Skip to content

UAbstractGymConnector

An abstract class for connectors between Unreal Engine and gym environments.

class provides the basic structure and functionality for connecting Unreal Engine environments with external gym environments for training purposes.

Subclassed by UExternalGymConnector, UManualGymConnector

class UAbstractGymConnector : public UObject

Methods

UAbstractGymConnector

UAbstractGymConnector()

Constructor for the abstract gym connector.

UAbstractGymConnector

~UAbstractGymConnector()

Destructor for the gym connector.

Init

virtual void Init(const FTrainingDefinition &AgentDefinitions)

Initialize this gym connector with training definitions.

Parameters

  • AgentDefinitions (const FTrainingDefinition)

Init

void Init(
    const TArray<TScriptInterface<IBaseScholaEnvironment>> &InEnvironments)

Initialize this gym connector by collecting and setting up environments.

Parameters

  • InEnvironments (const TArray<TScriptInterface<IBaseScholaEnvironment>>)

SetStatus

void SetStatus(EConnectorStatus NewStatus)

Set the operational status of the connector.

Parameters

  • NewStatus (EConnectorStatus)

SubmitState

inline virtual void SubmitState(const FTrainingState &InTrainingState)

Submit the current training state to the external training system.

Parameters

  • InTrainingState (const FTrainingState)

SubmitStateWithInitialState

inline virtual void
SubmitStateWithInitialState(const FTrainingState &InTrainingState,
                            const FInitialState &InInitialAgentStates)

Submit both training state and initial state (for environments that reset).

Parameters

  • InTrainingState (const FTrainingState)

  • InInitialAgentStates (const FInitialState)

SubmitInitialState

inline virtual void
SubmitInitialState(const FInitialState &InInitialAgentStates)

Submit only the initial state (for reset-only operations).

Parameters

  • InInitialAgentStates (const FInitialState)

HandleStep

void HandleStep(const FTrainingStep &InState, FTrainingState &OutTrainingState,
                FInitialState &OutInitialState)

Handle a step update by applying actions to environments.

Parameters

  • InState (const FTrainingStep)

  • OutTrainingState (FTrainingState)

  • OutInitialState (FInitialState)

HandleReset

void HandleReset(const FTrainingReset &InReset,
                 FTrainingState &OutTrainingState,
                 FInitialState &OutInitialState)

Handle a reset update by resetting specified environments.

Parameters

  • InReset (const FTrainingReset)

  • OutTrainingState (FTrainingState)

  • OutInitialState (FInitialState)

ResolveEnvironmentStateUpdate

inline virtual FTrainingStateUpdate *ResolveEnvironmentStateUpdate()

Resolve the environment state update.

UpdateConnectorStatus

void UpdateConnectorStatus(const FTrainingStateUpdate &StateUpdate)

Update the connector status based on a state update.

Parameters

  • StateUpdate (const FTrainingStateUpdate)

CheckForStart

inline virtual bool CheckForStart()

Check if the connector is ready to start.

IsRunning

inline bool IsRunning()

Get if the connector is running.

IsNotStarted

inline bool IsNotStarted()

Get if the connector is closed.

CollectEnvironments

void CollectEnvironments(
    TArray<TScriptInterface<IBaseScholaEnvironment>> &OutCollectedEnvironments)

Collect all environment objects in the current world.

Parameters

  • OutCollectedEnvironments (TArray<TScriptInterface<IBaseScholaEnvironment>>)

PrepareEnvironments

void PrepareEnvironments(
    const TArray<TScriptInterface<IBaseScholaEnvironment>> &InEnvironments)

Prepare environments for training by initializing and storing them.

Parameters

  • InEnvironments (const TArray<TScriptInterface<IBaseScholaEnvironment>>)

GetCompletedEnvironmentIds

void GetCompletedEnvironmentIds(const FTrainingState &InState,
                                TArray<int> &OutCompletedEnvironmentIds)

Get the IDs of environments that have completed their episodes.

Parameters

  • InState (const FTrainingState)

  • OutCompletedEnvironmentIds (TArray<int>)

Step

void Step()

Execute one training step across all environments.

Step

void Step(FTrainingState &OutTrainingState, FInitialState &OutInitialState)

Execute one training step and retrieve the resulting states.

Parameters

  • OutTrainingState (FTrainingState)

  • OutInitialState (FInitialState)

GetAutoResetType

inline virtual EAutoResetType GetAutoResetType()

Get the auto-reset behavior for this connector.

Attributes

OnConnectorStarted

FConnectorStartedSignature OnConnectorStarted

Delegate broadcast when the connector starts successfully.

OnConnectorClosed

FConnectorClosedSignature OnConnectorClosed

Delegate broadcast when the connector closes.

OnConnectorError

FConnectorErrorSignature OnConnectorError

Delegate broadcast when the connector encounters an error.

Environments

TArray<IScholaEnvironment *> Environments = TArray<IScholaEnvironment *>()

The environments that are currently being used for training by this connector.

FTrainingState

FTrainingState TrainingState = FTrainingState()

The current training state containing observations, rewards, and done flags for all environments.

FInitialState

FInitialState InitialState = FInitialState()

The initial state of environments at the start of the last episode.

FTrainingDefinition

FTrainingDefinition TrainingDefinition = FTrainingDefinition()

Definition of the training session including environment and agent specifications.

Status

EConnectorStatus Status = EConnectorStatus::Running

The current operational status of the connector.

bFirstStep

bool bFirstStep = true

Flag indicating if this is the first step after initialization.

bRunEnvironmentsInParallel

bool bRunEnvironmentsInParallel = false

Flag indicating if the environments should be run in parallel.

MinParallelBatchSize

int32 MinParallelBatchSize = 1

The minimum number of environments to process in each parallel batch.

InitialStateCriticalSection

FCriticalSection InitialStateCriticalSection

Critical section for protecting access to InitialState.EnvironmentStates during parallel operations.

Source: Source/ScholaTraining/Public/GymConnectors/AbstractGymConnector.h