Class UAbstractGymConnector

Class UAbstractGymConnector

• Defined in File AbstractGymConnector.h

Inheritance Relationships

Base Type

• public UObject

Derived Types

• public UExternalGymConnector (Class UExternalGymConnector)

• public UManualGymConnector (Class UManualGymConnector)

class UAbstractGymConnector : public UObject

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Dependencies: FInitialState, FTrainingDefinition, FTrainingReset, FTrainingState, FTrainingStateUpdate, FTrainingStep, IBaseScholaEnvironment, IScholaEnvironment

An abstract class for connectors between Unreal Engine and gym environments.

class provides the basic structure and functionality for connecting Unreal Engine environments with external gym environments for training purposes.

Subclassed by UExternalGymConnector, UManualGymConnector

---

Public Functions

Symbol	Details
UAbstractGymConnector	Constructor for the abstract gym connector.
~UAbstractGymConnector	Destructor for the gym connector.
Init	Initialize this gym connector with training definitions.
Init	Initialize this gym connector by collecting and setting up environments.
SetStatus	Set the operational status of the connector.
SubmitState	Submit the current training state to the external training system.
SubmitStateWithInitialState	Submit both training state and initial state (for environments that reset).
SubmitInitialState	Submit only the initial state (for reset-only operations).
HandleStep	Handle a step update by applying actions to environments.
HandleReset	Handle a reset update by resetting specified environments.
ResolveEnvironmentStateUpdate	Resolve the environment state update.
UpdateConnectorStatus	Update the connector status based on a state update.
CheckForStart	Check if the connector is ready to start.
IsRunning	Get if the connector is running.
IsNotStarted	Get if the connector is closed.
CollectEnvironments	Collect all environment objects in the current world.
PrepareEnvironments	Prepare environments for training by initializing and storing them.
GetCompletedEnvironmentIds	Get the IDs of environments that have completed their episodes.
Step	Execute one training step across all environments.
Step	Execute one training step and retrieve the resulting states.
GetAutoResetType	Get the auto-reset behavior for this connector.

UAbstractGymConnector

UAbstractGymConnector()

Constructor for the abstract gym connector.

Initializes the connector with default values.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

---

~UAbstractGymConnector

~UAbstractGymConnector()

Destructor for the gym connector.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

Init

virtual void Init(const FTrainingDefinition &AgentDefinitions)

Initialize this gym connector with training definitions.

Sets up communication channels and prepares the connector for training. Override this in derived classes to implement specific initialization logic.

Parameters

AgentDefinitions – [in] The definitions of the agents and environments that will be trained.

#	Direction	Name	Type	Description
1	—	AgentDefinitions	const FTrainingDefinition &	The definitions of the agents and environments that will be trained.

Attributes: virtual

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

Init

void Init(
    const TArray<TScriptInterface<IBaseScholaEnvironment>> &InEnvironments)

Initialize this gym connector by collecting and setting up environments.

Gathers environment definitions and calls the other Init function.

Parameters

InEnvironments – [in] Array of environment interfaces to train.

#	Direction	Name	Type	Description
1	—	AgentDefinitions	const FTrainingDefinition &	The definitions of the agents and environments that will be trained.

Attributes: virtual

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

SetStatus

void SetStatus(EConnectorStatus NewStatus)

Set the operational status of the connector.

Also broadcasts appropriate delegates when status changes.

Parameters

NewStatus – [in] The new status to set.

#	Direction	Name	Type	Description
1	—	NewStatus	EConnectorStatus	The new status to set.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

SubmitState

inline virtual void SubmitState(const FTrainingState &InTrainingState)

Submit the current training state to the external training system.

Must be implemented by derived classes to handle state submission.

Parameters

InTrainingState – [in] The state containing observations, rewards, and done flags.

#	Direction	Name	Type	Description
1	—	InTrainingState	const FTrainingState &	The state containing observations, rewards, and done flags.

Attributes: inline, virtual

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

SubmitStateWithInitialState

inline virtual void
SubmitStateWithInitialState(const FTrainingState &InTrainingState,
                            const FInitialState &InInitialAgentStates)

Submit both training state and initial state (for environments that reset).

Must be implemented by derived classes.

Parameters

• InTrainingState – [in] The current training state.

• InInitialAgentStates – [in] The initial states for reset environments.

#	Direction	Name	Type	Description
1	—	InTrainingState	const FTrainingState &	The current training state.
2	—	InInitialAgentStates	const FInitialState &	The initial states for reset environments.

Attributes: inline, virtual

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

SubmitInitialState

inline virtual void
SubmitInitialState(const FInitialState &InInitialAgentStates)

Submit only the initial state (for reset-only operations).

Must be implemented by derived classes.

Parameters

InInitialAgentStates – [in] The initial states for reset environments.

#	Direction	Name	Type	Description
1	—	InInitialAgentStates	const FInitialState &	The initial states for reset environments.

Attributes: inline, virtual

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

HandleStep

void HandleStep(const FTrainingStep &InState, FTrainingState &OutTrainingState,
                FInitialState &OutInitialState)

Handle a step update by applying actions to environments.

Parameters

• InState – [in] The step data containing actions for each environment.

• OutTrainingState – [out] The resulting training state after the step.

• OutInitialState – [out] The initial states for any environments that reset.

#	Direction	Name	Type	Description
1	—	InState	const FTrainingStep &	The step data containing actions for each environment.
2	—	OutTrainingState	FTrainingState &	The resulting training state after the step.
3	—	OutInitialState	FInitialState &	The initial states for any environments that reset.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

HandleReset

void HandleReset(const FTrainingReset &InReset,
                 FTrainingState &OutTrainingState,
                 FInitialState &OutInitialState)

Handle a reset update by resetting specified environments.

Parameters

• InReset – [in] The reset data specifying which environments to reset.

• OutTrainingState – [out] The resulting training state (empty for pure resets).

• OutInitialState – [out] The initial states after reset.

#	Direction	Name	Type	Description
1	—	InReset	const FTrainingReset &	The reset data specifying which environments to reset.
2	—	OutTrainingState	FTrainingState &	The resulting training state (empty for pure resets).
3	—	OutInitialState	FInitialState &	The initial states after reset.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

ResolveEnvironmentStateUpdate

inline virtual FTrainingStateUpdate *ResolveEnvironmentStateUpdate()

Resolve the environment state update.

Useful for connections that operate off of futures, or otherwise require synchronization.

Note

This function should be implemented by a derived class.

Returns:

The resolved environment state update.

Attributes: inline, virtual

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

UpdateConnectorStatus

void UpdateConnectorStatus(const FTrainingStateUpdate &StateUpdate)

Update the connector status based on a state update.

Parameters

StateUpdate – [in] The state update to base the new status on.

#	Direction	Name	Type	Description
1	—	StateUpdate	const FTrainingStateUpdate &	The state update to base the new status on.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

CheckForStart

inline virtual bool CheckForStart()

Check if the connector is ready to start.

Note

This function should be implemented by a derived class.

Returns:

True if the connector is ready to start.

Attributes: inline, virtual

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

IsRunning

inline bool IsRunning()

Get if the connector is running.

Returns:

True if the connector is running.

Attributes: inline

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

IsNotStarted

inline bool IsNotStarted()

Get if the connector is closed.

Returns:

True if the connector is closed.

Attributes: inline

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

CollectEnvironments

void CollectEnvironments(
    TArray<TScriptInterface<IBaseScholaEnvironment>> &OutCollectedEnvironments)

Collect all environment objects in the current world.

Searches the level for all objects implementing IBaseScholaEnvironment.

Parameters

OutCollectedEnvironments – [out] Array of found environment interfaces.

#	Direction	Name	Type	Description
1	—	OutCollectedEnvironments	TArray< TScriptInterface< IBaseScholaEnvironment > > &	Array of found environment interfaces.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

PrepareEnvironments

void PrepareEnvironments(
    const TArray<TScriptInterface<IBaseScholaEnvironment>> &InEnvironments)

Prepare environments for training by initializing and storing them.

Converts environment interfaces to the internal type-erased format.

Parameters

InEnvironments – [in] Array of environment interfaces to prepare.

#	Direction	Name	Type	Description
1	—	InEnvironments	const TArray< TScriptInterface< IBaseScholaEnvironment > > &	Array of environment interfaces to prepare.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

GetCompletedEnvironmentIds

void GetCompletedEnvironmentIds(const FTrainingState &InState,
                                TArray<int> &OutCompletedEnvironmentIds)

Get the IDs of environments that have completed their episodes.

Parameters

• InState – [in] The training state to check for completed environments.

• OutCompletedEnvironmentIds – [out] Array of environment IDs that are done.

#	Direction	Name	Type	Description
1	—	InState	const FTrainingState &	The training state to check for completed environments.
2	—	OutCompletedEnvironmentIds	TArray< int > &	Array of environment IDs that are done.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

Step

void Step()

Execute one training step across all environments.

Retrieves the next action/reset update and applies it to environments. Internal state is updated in place.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

Step

void Step(FTrainingState &OutTrainingState, FInitialState &OutInitialState)

Execute one training step and retrieve the resulting states.

Parameters

• OutTrainingState – [out] The training state after the step.

• OutInitialState – [out] The initial states for any reset environments.

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

Implementation: Source/Schola/Training/Private/GymConnectors/AbstractGymConnector.cpp

GetAutoResetType

inline virtual EAutoResetType GetAutoResetType()

Get the auto-reset behavior for this connector.

Controls when environments automatically reset after episode completion.

Returns:

The auto-reset type (default: SameStep).

Attributes: inline, virtual

Source: Source/Schola/Training/Public/GymConnectors/AbstractGymConnector.h

---

Public Members

Symbol	Details
OnConnectorStarted	Delegate broadcast when the connector starts successfully.
OnConnectorClosed	Delegate broadcast when the connector closes.
OnConnectorError	Delegate broadcast when the connector encounters an error.
Environments	The environments that are currently being used for training by this connector.
TrainingState	The current training state containing observations, rewards, and done flags for all environments.
InitialState	The initial state of environments at the start of the last episode.
TrainingDefinition	Definition of the training session including environment and agent specifications.
Status	The current operational status of the connector.
bFirstStep	Flag indicating if this is the first step after initialization.

---

OnConnectorStarted

FConnectorStartedSignature OnConnectorStarted

Delegate broadcast when the connector starts successfully.

---

OnConnectorClosed

FConnectorClosedSignature OnConnectorClosed

Delegate broadcast when the connector closes.

---

OnConnectorError

FConnectorErrorSignature OnConnectorError

Delegate broadcast when the connector encounters an error.

---

Environments

TArray<IScholaEnvironment *> Environments = TArray<IScholaEnvironment *>()

The environments that are currently being used for training by this connector.

---

TrainingState

FTrainingState TrainingState = FTrainingState()

The current training state containing observations, rewards, and done flags for all environments.

---

InitialState

FInitialState InitialState = FInitialState()

The initial state of environments at the start of the last episode.

---

TrainingDefinition

FTrainingDefinition TrainingDefinition = FTrainingDefinition()

Definition of the training session including environment and agent specifications.

---

Status

EConnectorStatus Status = EConnectorStatus::Running

The current operational status of the connector.

---

bFirstStep

bool bFirstStep = true

Flag indicating if this is the first step after initialization.