schola.scripts.ray.settings.PPOSettings

class schola.scripts.ray.settings.PPOSettings(gae_lambda=0.95, clip_param=0.2, use_gae=True)[source]

Bases: RLLibAlgorithmSpecificSettings

Dataclass for PPO (Proximal Policy Optimization) algorithm specific settings. This class defines the parameters used in the PPO algorithm, including GAE lambda, clip parameter, and whether to use GAE.

Methods

__init__([gae_lambda, clip_param, use_gae])
get_parser()	Add the settings to the parser or subparser
get_settings_dict()	Get the settings as a dictionary keyed by the correct parameter name in Ray

Attributes

clip_param	The clip parameter for the PPO algorithm.
gae_lambda	The lambda parameter for Generalized Advantage Estimation (GAE).
name
rllib_config
use_gae	Whether to use Generalized Advantage Estimation (GAE) for advantage calculation.

Parameters:

• gae_lambda (float)

• clip_param (float)

• use_gae (bool)

__init__(gae_lambda=0.95, clip_param=0.2, use_gae=True)

Parameters:

• gae_lambda (float)

• clip_param (float)

• use_gae (bool)

Return type:

None

clip_param: float = 0.2

The clip parameter for the PPO algorithm. This is the epsilon value used in the clipped surrogate objective function. It helps to limit the policy update step size to prevent large changes that could lead to performance collapse.

gae_lambda: float = 0.95

The lambda parameter for Generalized Advantage Estimation (GAE). This controls the trade-off between bias and variance in the advantage estimation.

classmethod get_parser()[source]

Add the settings to the parser or subparser

get_settings_dict()[source]

Get the settings as a dictionary keyed by the correct parameter name in Ray

property name: str

property rllib_config: Type[PPOConfig]

use_gae: bool = True

Whether to use Generalized Advantage Estimation (GAE) for advantage calculation. GAE is a method to reduce the variance of the advantage estimates while keeping bias low. If set to False, the standard advantage calculation will be used instead.