Skip to content

PPOSettings

Full path: schola.scripts.rllib.settings.PPOSettings

Dataclass for PPO (Proximal Policy Optimization) algorithm specific settings. This class defines the parameters used in the PPO algorithm, including GAE lambda, clip parameter, and whether to use GAE.

PPOSettings(gae_lambda = 0.95, clip_param = 0.2, use_gae = True)

Bases: RllibAlgorithmSpecificSettings

Parameters

  • gae_lambda (Annotated)

  • clip_param (Annotated)

  • use_gae (bool)

Methods

init

__init__(gae_lambda = 0.95, clip_param = 0.2, use_gae = True)

Parameters

  • gae_lambda (Annotated)

  • clip_param (Annotated)

  • use_gae (bool)

get_settings_dict

get_settings_dict()

Get the settings as a dictionary keyed by the correct parameter name in Ray

Attributes

clip_param

clip_param

The clip parameter for the PPO algorithm. This is the epsilon value used in the clipped surrogate objective function. It helps to limit the policy update step size to prevent large changes that could lead to performance collapse.

gae_lambda

gae_lambda

The lambda parameter for Generalized Advantage Estimation (GAE). This controls the trade-off between bias and variance in the advantage estimation.

use_gae

use_gae

Whether to use Generalized Advantage Estimation (GAE) for advantage calculation. GAE is a method to reduce the variance of the advantage estimates while keeping bias low. If set to False, the standard advantage calculation will be used instead.

name

name

rllib_config

rllib_config