AAbstractTrainer

class AAbstractTrainer : public AController

An abstract class representing a controller that trains an NPC using Reinforcement Learning.

Note: This class is designed to be subclassed in C++ or Blueprint to implement the specific training logic for an NPC.

Note: This class is designed to be used in conjunction with the AbstractEnvironment class.

Dependencies

This type depends on:

Inherits from: public AController

Subclassed by: ABlueprintTrainer

Public Interface

Constructor:

AAbstractTrainer

AAbstractTrainer()

Construct a new AAbstractTrainer object.

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 50, column 1)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 55-58)

Public Functions:

PawnPendingDestroy

virtual void PawnPendingDestroy(APawn *inPawn) override

Parameters:

Name	Type	Default
`inPawn`	`APawn *`	“

Attributes: virtual

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 45, column 14)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 29-46)

Initialize

bool Initialize(int EnvId, int AgentId, APawn *TargetPawn)

Initialize this agent after play has begun.

Parameters:

EnvId (int) – [in] The ID of the environment this agent is in.
AgentId (int) – [in] The ID of this agent in the environment.
TargetPawn (APawn *) – [in] A Pawn of the Class that this Trainer will manage. Can be a CDO or a pawn in the scene.

Returns: bool

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 84, column 6)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 60-119)

ComputeReward

inline virtual float ComputeReward()

Collect a reward from the agent’s immediate environment.

Returns: float representing the agents reward

Note: This function must be implemented by a subclass.

Returns: float

Attributes: inline, virtual

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 91, column 15)

ComputeStatus

inline virtual EAgentTrainingStatus ComputeStatus()

Check if agent is in a terminal state.

Returns: The status of the agent which informs whether it is still running, or why it stopped.

Note: This function must be implemented by a subclass.

Returns: EAgentTrainingStatus

Attributes: inline, virtual

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 97, column 30)

GetInfo

inline virtual void GetInfo(TMap<FString, FString> &Info)

Get the Info output of the agent.

Parameters:

Info (TMap<FString, FString> &) – [out] A mapping representing non-observation details about the environment.

Attributes: inline, virtual

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 103, column 14)

GetTrainingStatus

EAgentTrainingStatus GetTrainingStatus()

Get the last computed training status.

Returns: The last computed training status.

Returns: EAgentTrainingStatus

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 109, column 22)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 184-187)

GetTrainingMsgStatus

ETrainingMsgStatus GetTrainingMsgStatus()

Get whether the agent has finished, and if it has sent a final state update noting that it is finished.

Returns: An enum tracking whether the environment has sent a final state update after ending.

Returns: ETrainingMsgStatus

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 115, column 20)

SetTrainingStatus

void SetTrainingStatus(EAgentTrainingStatus NewStatus)

Set the Agent’s Training Status.

Parameters:

NewStatus (EAgentTrainingStatus) – [in] The new status to set

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 121, column 6)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 178-182)

SetTrainingMsgStatus

void SetTrainingMsgStatus(ETrainingMsgStatus NewStatus)

Set the Agent’s Training Message Status.

Parameters:

NewStatus (ETrainingMsgStatus) – [in] The new status to set

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 127, column 6)

IsDone

bool IsDone() const const

Does this agent need resetting (either Truncated or Complete)

Returns: true iff the agent needs resetting

Returns: bool

Attributes: const

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 134, column 6)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 194-197)

Reset

void Reset()

Reset the agent.

Collect initial observations of the environment after resetting in the process.

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 140, column 6)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 165-176)

ResetTrainer

inline virtual void ResetTrainer()

Reset any per Episode properties of this Trainer.

Note: This function must be implemented by a subclass.

Attributes: inline, virtual

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 146, column 14)

IncrementStep

inline void IncrementStep()

increment the step count for this episode.

This is used to determine when to request a new action from the brain.

Attributes: inline

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 151, column 6)

ResetStep

inline void ResetStep()

Reset the step count to 0.

This is used to determine when to request a new action from the brain.

Attributes: inline

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 156, column 6)

GetStep

inline int GetStep() const const

Get the step of the agent in the current episode.

Returns: int

Attributes: const, inline

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 162, column 5)

SetStep

inline void SetStep(int InStep)

Parameters:

Name	Type	Default
`InStep`	`int`	“

Attributes: inline

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 165, column 6)

IsDecisionStep

virtual bool IsDecisionStep(int StepToCheck)

Check whether a specific step will require a brain decision.

Returns: true iff the agent should be requesting a decision

Parameters:

StepToCheck (int) – the timestep to check

Returns: bool

Attributes: virtual

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 172, column 14)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 204-207)

IsDecisionStep

virtual bool IsDecisionStep()

If the current step is a decision step, as defined by the step frequency.

Returns: true iff the current step is a decision step

Returns: bool

Attributes: virtual

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 178, column 14)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 209-212)

IsActionStep

virtual bool IsActionStep()

Check if brain has an action, and it’s an action step.

Returns: true iff the agent should take an action this step

Returns: bool

Attributes: virtual

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 184, column 14)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 199-202)

Act

void Act(const FAction &Action)

The Agent retrieves an action from the brain before taking it.

Parameters:

Action (const FAction &) – [in] The action to take

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 190, column 6)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 158-163)

Think

FTrainerState * Think()

Update the state of the agent.

This checks if the agent is done, what it’s reward should be and does any observation collection before requesting a decision

Returns: The state of the agent after the update

Returns: FTrainerState *

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 196, column 15)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 121-156)

IsRunning

bool IsRunning()

Check with brain if can act and set agent state accordingly.

Returns: The state of the agent after the update

Returns: bool

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 202, column 6)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 189-192)

OnCompletion

inline virtual void OnCompletion()

Callback function for logic when agent completes its episode.

Attributes: inline, virtual

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 207, column 14)

HasAgentClass

inline bool HasAgentClass() const const

Returns: bool

Attributes: const, inline

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 209, column 6)

Public Members:

FTrainerState State

FTrainerState State = = FTrainerState()

The current state of the agent.

Memory is managed by the SharedState inside the GymConnector

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 53, column 15)

UInteractionManager * InteractionManager

UInteractionManager * InteractionManager = = CreateDefaultSubobject<UInteractionManager>(TEXT("InteractionManager"))

An Object for managing the Interactions of the agent and the environment.

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 57, column 21)

TArray<UAbstractObserver *> Observers

TArray<UAbstractObserver *> Observers

List of observers that collect observations for the agent.

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 61, column 8)

TArray<UActuator *> Actuators

TArray<UActuator *> Actuators

List of actuators that execute actions for the agent.

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 65, column 8)

FTrainerConfiguration TrainerConfiguration

FTrainerConfiguration TrainerConfiguration

The configuration of the Trainer.

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 69, column 23)

FTrainerDefinition TrainerDefn

FTrainerDefinition TrainerDefn

The definition of the agent.

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 73, column 20)

TSubclassOf<APawn> AgentClass

TSubclassOf<APawn> AgentClass

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 76, column 13)

Protected Interface

Protected Functions:

OnPossess

void OnPossess(APawn *InPawn) override

Parameters:

Name	Type	Default
`InPawn`	`APawn *`	“

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 39, column 6)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 9-26)

OnUnPossess

void OnUnPossess() override

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 41, column 6)

Implementation: Schola/Source/Schola/Private/Training/AbstractTrainer.cpp (lines 48-53)

Used By

This type is used by:

Source: Schola/Source/Schola/Public/Training/AbstractTrainer.h (line 32, column 1)