APPOSettings

Full path: schola.scripts.rllib.settings.APPOSettings

schola.scripts.rllib.settings.APPOSettings

APPOSettings

APPOSettings(
    gae_lambda=0.95,
    clip_param=0.2,
    use_gae=True,
    vtrace=True,
    vtrace_clip_rho_threshold=1.0,
    vtrace_clip_pg_rho_threshold=1.0,
)

Bases: IMPALASettings, PPOSettings

Dataclass for APPO (Asynchronous Proximal Policy Optimization) algorithm specific settings. This class inherits from both IMPALASettings and PPOSettings to combine the settings for both algorithms. This allows for the use of both V-trace for off-policy correction and PPO for policy optimization in a single algorithm.

Methods

Item	Description
init	—
`get_parser`()	Add the settings to the parser or subparser
get_settings_dict	Get the settings as a dictionary keyed by the correct parameter name in Ray

Attributes

Item	Description
`clip_param`	The clip parameter for the PPO algorithm.
`gae_lambda`	The lambda parameter for Generalized Advantage Estimation (GAE).
name	—
rllib_config	—
`use_gae`	Whether to use Generalized Advantage Estimation (GAE) for advantage calculation.
`vtrace`	Whether to use the V-trace algorithm for off-policy correction in the IMPALA algorithm.
`vtrace_clip_pg_rho_threshold`	The clip threshold for V-trace rho values in the policy gradient.
`vtrace_clip_rho_threshold`	The clip threshold for V-trace rho values.

Parameters

gae_lambda (Annotated[float, Parameter(validator=(Number(lt=None, lte=1.0, gt=None, gte=0.0, modulo=None),))])

clip_param (Annotated[float, Parameter(validator=(Number(lt=None, lte=None, gt=None, gte=0, modulo=None),))])

use_gae (bool)

vtrace (bool)

vtrace_clip_rho_threshold (Annotated[float, Parameter(validator=(Number(lt=None, lte=None, gt=None, gte=0, modulo=None),))])

vtrace_clip_pg_rho_threshold (Annotated[float, Parameter(validator=(Number(lt=None, lte=None, gt=None, gte=0, modulo=None),))])

init

__init__(
    gae_lambda=0.95,
    clip_param=0.2,
    use_gae=True,
    vtrace=True,
    vtrace_clip_rho_threshold=1.0,
    vtrace_clip_pg_rho_threshold=1.0,
)

Parameters

gae_lambda (Annotated[float, Parameter(validator=(Number(lt=None, lte=1.0, gt=None, gte=0.0, modulo=None),))])

clip_param (Annotated[float, Parameter(validator=(Number(lt=None, lte=None, gt=None, gte=0, modulo=None),))])

use_gae (bool)

vtrace (bool)

vtrace_clip_rho_threshold (Annotated[float, Parameter(validator=(Number(lt=None, lte=None, gt=None, gte=0, modulo=None),))])

vtrace_clip_pg_rho_threshold (Annotated[float, Parameter(validator=(Number(lt=None, lte=None, gt=None, gte=0, modulo=None),))])

Returns

None

get_settings_dict

get_settings_dict()

Get the settings as a dictionary keyed by the correct parameter name in Ray

name

name: str

rllib_config

rllib_config: Type[APPOConfig]