Class AAbstractTrainer
- class AAbstractTrainer : public AController
-
An abstract class representing a controller that trains an NPC using Reinforcement Learning.
Note
This class is designed to be subclassed in C++ or Blueprint to implement the specific training logic for an NPC.
Note
This class is designed to be used in conjunction with the AbstractEnvironment class.
Subclassed by ABlueprintTrainer
Public Functions
- AAbstractTrainer()
-
Construct a new AAbstractTrainer object.
- bool Initialize(int EnvId, int AgentId)
-
Initialize this agent after play has begun.
- Parameters:
-
-
EnvId – [in] The ID of the environment this agent is in.
-
AgentId – [in] The ID of this agent in the environment.
-
- inline virtual float ComputeReward()
-
Collect a reward from the agent’s immediate environment.
Note
This function must be implemented by a subclass.
- Returns:
-
float representing the agents reward
- inline virtual EAgentTrainingStatus ComputeStatus()
-
Check if agent is in a terminal state.
Note
This function must be implemented by a subclass.
- Returns:
-
The status of the agent which informs whether it is still running, or why it stopped.
- inline virtual void GetInfo(TMap<FString, FString> &Info)
-
Get the Info output of the agent.
- Parameters:
-
Info – [out] A mapping representing non-observation details about the environment.
- EAgentTrainingStatus GetTrainingStatus()
-
Get the last computed training status.
- Returns:
-
The last computed training status.
- ETrainingMsgStatus GetTrainingMsgStatus()
-
Get whether the agent has finished, and if it has sent a final state update noting that it is finished.
- Returns:
-
An enum tracking whether the environment has sent a final state update after ending.
- void SetTrainingStatus(EAgentTrainingStatus NewStatus)
-
Set the Agent’s Training Status.
- Parameters:
-
NewStatus – [in] The new status to set
- void SetTrainingMsgStatus(ETrainingMsgStatus NewStatus)
-
Set the Agent’s Training Message Status.
- Parameters:
-
NewStatus – [in] The new status to set
- bool IsDone() const
-
Does this agent need resetting (either Truncated or Complete)
- Returns:
-
true iff the agent needs resetting
- void Reset()
-
Reset the agent.
Collect initial observations of the environment after resetting in the process.
- inline virtual void ResetTrainer()
-
Reset any per Episode properties of this Trainer.
Note
This function must be implemented by a subclass.
- inline void IncrementStep()
-
increment the step count for this episode.
This is used to determine when to request a new action from the brain.
- inline void ResetStep()
-
Reset the step count to 0.
This is used to determine when to request a new action from the brain.
- virtual bool IsDecisionStep(int StepToCheck)
-
Check whether a specific step will require a brain decision.
- Parameters:
-
StepToCheck – the timestep to check
- Returns:
-
true iff the agent should be requesting a decision
- virtual bool IsDecisionStep()
-
If the current step is a decision step, as defined by the step frequency.
- Returns:
-
true iff the current step is a decision step
- virtual bool IsActionStep()
-
Check if brain has an action, and it’s an action step.
- Returns:
-
true iff the agent should take an action this step
- void Act(const FAction &Action)
-
The Agent retrieves an action from the brain before taking it.
- Parameters:
-
Action – [in] The action to take
- FTrainerState Think()
-
Update the state of the agent.
This checks if the agent is done, what it’s reward should be and does any observation collection before requesting a decision
- Returns:
-
The state of the agent after the update
Public Members
- FTrainerState State = FTrainerState()
-
The current state of the agent.
- UInteractionManager *InteractionManager = CreateDefaultSubobject<UInteractionManager>(TEXT(“InteractionManager”))
-
An Object for managing the Interactions of the agent and the environment.
- TArray<UAbstractObserver*> Observers
-
List of observers that collect observations for the agent.
- TArray<UActuator*> Actuators
-
List of actuators that execute actions for the agent.
- EValidationType Validation = EValidationType::FAIL
-
The type of validation to perform on this agent.
Fail means agent is skipped on any error. Warning means just warn about non-fatal errors. Set to No Validation to skip validation
- bool bTakeActionBetweenDecisions = true
-
If true the agent will repeat its last action each step between decision requests.
- int DecisionRequestFrequency = 5
-
The number of steps between requests for new actions.
If this is different across agents it may cause issues training in some frameworks (e.g. Stable Baselines 3).
- bool bAbstractSettingsVisibility = true
-
Whether settings in this class are visible in the editor for child classes.
Set to False to hide those settings.
- FTrainerDefinition TrainerDefn
-
The definition of the agent.
