schola.scripts.ray.settings.APPOSettings

class schola.scripts.ray.settings.APPOSettings(gae_lambda=0.95, clip_param=0.2, use_gae=True, vtrace=True, vtrace_clip_rho_threshold=1.0, vtrace_clip_pg_rho_threshold=1.0)[source]

Bases: IMPALASettings, PPOSettings

Dataclass for APPO (Asynchronous Proximal Policy Optimization) algorithm specific settings. This class inherits from both IMPALASettings and PPOSettings to combine the settings for both algorithms. This allows for the use of both V-trace for off-policy correction and PPO for policy optimization in a single algorithm.

Methods

__init__([gae_lambda, clip_param, use_gae, …])
get_parser()	Add the settings to the parser or subparser
get_settings_dict()	Get the settings as a dictionary keyed by the correct parameter name in Ray

Attributes

clip_param	The clip parameter for the PPO algorithm.
gae_lambda	The lambda parameter for Generalized Advantage Estimation (GAE).
name
rllib_config
use_gae	Whether to use Generalized Advantage Estimation (GAE) for advantage calculation.
vtrace	Whether to use the V-trace algorithm for off-policy correction in the IMPALA algorithm.
vtrace_clip_pg_rho_threshold	The clip threshold for V-trace rho values in the policy gradient.
vtrace_clip_rho_threshold	The clip threshold for V-trace rho values.

Parameters:

• gae_lambda (float)

• clip_param (float)

• use_gae (bool)

• vtrace (bool)

• vtrace_clip_rho_threshold (float)

• vtrace_clip_pg_rho_threshold (float)

__init__(gae_lambda=0.95, clip_param=0.2, use_gae=True, vtrace=True, vtrace_clip_rho_threshold=1.0, vtrace_clip_pg_rho_threshold=1.0)

Parameters:

• gae_lambda (float)

• clip_param (float)

• use_gae (bool)

• vtrace (bool)

• vtrace_clip_rho_threshold (float)

• vtrace_clip_pg_rho_threshold (float)

Return type:

None

classmethod get_parser()[source]

Add the settings to the parser or subparser

get_settings_dict()[source]

Get the settings as a dictionary keyed by the correct parameter name in Ray

property name: str

property rllib_config: Type[APPOConfig]