Skip to content

SACSettings

Full path: schola.scripts.rllib.settings.SACSettings

Dataclass for SAC (Soft Actor-Critic) algorithm specific settings. This class defines the parameters used in the SAC algorithm, including soft target network updates and entropy regularization.

SACSettings(tau = 0.005, target_entropy = 'auto', initial_alpha = 1.0,
            n_step = 1, twin_q = True)

Bases: RllibAlgorithmSpecificSettings

Parameters

  • tau (Annotated)

  • target_entropy (str)

  • initial_alpha (Annotated)

  • n_step (Annotated)

  • twin_q (bool)

Methods

init

__init__(tau = 0.005, target_entropy = 'auto', initial_alpha = 1.0, n_step = 1,
         twin_q = True)

Parameters

  • tau (Annotated)

  • target_entropy (str)

  • initial_alpha (Annotated)

  • n_step (Annotated)

  • twin_q (bool)

get_settings_dict

get_settings_dict()

Get the settings as a dictionary keyed by the correct parameter name in Ray

Attributes

initial_alpha

initial_alpha

Initial temperature/alpha value for entropy regularization. Higher values encourage more exploration.

n_step

n_step

Number of steps for n-step returns. Using n>1 can help with credit assignment in sparse reward environments.

target_entropy

target_entropy

Target entropy for automatic temperature tuning. Set to ‘auto’ to automatically calculate based on action space dimensionality, or provide a float value for manual control.

tau

tau

Soft update coefficient for target networks. Controls how quickly target networks track the main networks. Lower values (e.g., 0.005) mean slower updates, which can improve stability.

twin_q

twin_q

Whether to use twin Q networks (double Q-learning). This helps reduce overestimation bias in Q-value estimates.

name

name

rllib_config

rllib_config