Class AAbstractTrainer

class AAbstractTrainer : public AController

An abstract class representing a controller that trains an NPC using Reinforcement Learning.

Note

This class is designed to be subclassed in C++ or Blueprint to implement the specific training logic for an NPC.

Note

This class is designed to be used in conjunction with the AbstractEnvironment class.

Subclassed by ABlueprintTrainer

Public Functions

virtual void PawnPendingDestroy(APawn *inPawn) override

AAbstractTrainer()

Construct a new AAbstractTrainer object.

bool Initialize(int EnvId, int AgentId, APawn *TargetPawn)

Initialize this agent after play has begun.

Parameters:

• EnvId – [in] The ID of the environment this agent is in.

• AgentId – [in] The ID of this agent in the environment.

• TargetPawn – [in] A Pawn of the Class that this Trainer will manage. Can be a CDO or a pawn in the scene.

inline virtual float ComputeReward()

Collect a reward from the agent’s immediate environment.

Note

This function must be implemented by a subclass.

Returns:

float representing the agents reward

inline virtual EAgentTrainingStatus ComputeStatus()

Check if agent is in a terminal state.

Note

This function must be implemented by a subclass.

Returns:

The status of the agent which informs whether it is still running, or why it stopped.

inline virtual void GetInfo(TMap<FString, FString> &Info)

Get the Info output of the agent.

Parameters:

Info – [out] A mapping representing non-observation details about the environment.

EAgentTrainingStatus GetTrainingStatus()

Get the last computed training status.

Returns:

The last computed training status.

ETrainingMsgStatus GetTrainingMsgStatus()

Get whether the agent has finished, and if it has sent a final state update noting that it is finished.

Returns:

An enum tracking whether the environment has sent a final state update after ending.

void SetTrainingStatus(EAgentTrainingStatus NewStatus)

Set the Agent’s Training Status.

Parameters:

NewStatus – [in] The new status to set

void SetTrainingMsgStatus(ETrainingMsgStatus NewStatus)

Set the Agent’s Training Message Status.

Parameters:

NewStatus – [in] The new status to set

bool IsDone() const

Does this agent need resetting (either Truncated or Complete)

Returns:

true iff the agent needs resetting

void Reset()

Reset the agent.

Collect initial observations of the environment after resetting in the process.

inline virtual void ResetTrainer()

Reset any per Episode properties of this Trainer.

Note

This function must be implemented by a subclass.

inline void IncrementStep()

increment the step count for this episode.

This is used to determine when to request a new action from the brain.

inline void ResetStep()

Reset the step count to 0.

This is used to determine when to request a new action from the brain.

inline int GetStep() const

Get the step of the agent in the current episode.

inline void SetStep(int InStep)

virtual bool IsDecisionStep(int StepToCheck)

Check whether a specific step will require a brain decision.

Parameters:

StepToCheck – the timestep to check

Returns:

true iff the agent should be requesting a decision

virtual bool IsDecisionStep()

If the current step is a decision step, as defined by the step frequency.

Returns:

true iff the current step is a decision step

virtual bool IsActionStep()

Check if brain has an action, and it’s an action step.

Returns:

true iff the agent should take an action this step

void Act(const FAction &Action)

The Agent retrieves an action from the brain before taking it.

Parameters:

Action – [in] The action to take

FTrainerState *Think()

Update the state of the agent.

This checks if the agent is done, what it’s reward should be and does any observation collection before requesting a decision

Returns:

The state of the agent after the update

bool IsRunning()

Check with brain if can act and set agent state accordingly.

Returns:

The state of the agent after the update

inline virtual void OnCompletion()

Callback function for logic when agent completes its episode.

inline bool HasAgentClass() const

Public Members

FTrainerState State = FTrainerState()

The current state of the agent.

Memory is managed by the SharedState inside the GymConnector

UInteractionManager *InteractionManager = CreateDefaultSubobject<UInteractionManager>(TEXT(“InteractionManager”))

An Object for managing the Interactions of the agent and the environment.

TArray<UAbstractObserver*> Observers

List of observers that collect observations for the agent.

TArray<UActuator*> Actuators

List of actuators that execute actions for the agent.

FTrainerConfiguration TrainerConfiguration

The configuration of the Trainer.

FTrainerDefinition TrainerDefn

The definition of the agent.

TSubclassOf<APawn> AgentClass