Skip to content

UNNEPolicy

Policy implementation that uses trained NNE models for decision-making.

This policy class integrates Unreal Engine’s Neural Network Engine (NNE) with the Schola reinforcement learning framework, enabling trained neural network models to control agents. Supports both CPU and GPU inference runtimes.

class UNNEPolicy : public UObject, public IPolicy

Methods

Public

GetRuntimeNames

TArray<FString> GetRuntimeNames() const

Retrieves all available NNE runtime names.

GetRuntime

IRuntimeInterface *GetRuntime(const FString &SelectedRuntimeName) const

Gets the singleton runtime instance with the specified name.

Parameters

  • SelectedRuntimeName (const FString)

Think

inline bool Think(const FInstancedStruct &InObservations,
                  FInstancedStruct &OutAction)

Blueprint-callable wrapper for the Think function.

Parameters

  • InObservations (const FInstancedStruct)

  • OutAction (FInstancedStruct)

Think

virtual bool Think(const TInstancedStruct<FPoint> &InObservations,
                   TInstancedStruct<FPoint> &OutActions) override

Runs neural network inference to compute an action from observations.

Parameters

  • InObservations (const TInstancedStruct<FPoint>)

  • OutActions (TInstancedStruct<FPoint>)

Init

virtual bool Init(const FInteractionDefinition &InPolicyDefinition) override

Initializes the policy with the given interaction definition.

Parameters

  • InPolicyDefinition (const FInteractionDefinition)

IsInferenceBusy

virtual bool IsInferenceBusy() const override

Checks if an inference operation is currently running.

Protected

InitInputTensorShapes

bool InitInputTensorShapes(TSharedPtr<IModelInstanceRunSync> InModelInstance)

Initializes the input tensor shapes for the model instance.

Parameters

  • InModelInstance (TSharedPtr<IModelInstanceRunSync>)

AllocateBindingArrays

bool AllocateBindingArrays(TSharedPtr<IModelInstanceRunSync> InModelInstance)

Allocates arrays for input and output tensor bindings.

Parameters

  • InModelInstance (TSharedPtr<IModelInstanceRunSync>)

InitStateBuffersAndBindings

bool InitStateBuffersAndBindings(
    TSharedPtr<IModelInstanceRunSync> InModelInstance)

Initializes state buffers and their bindings for recurrent models.

Parameters

  • InModelInstance (TSharedPtr<IModelInstanceRunSync>)

InitNonStateBuffers

void InitNonStateBuffers(const FInteractionDefinition &InDefinition)

Initializes buffers for non-state data (observations and actions)

Parameters

  • InDefinition (const FInteractionDefinition)

InitNonStateBindings

bool InitNonStateBindings(const FInteractionDefinition &InDefinition)

Initializes tensor bindings for non-state data.

Parameters

  • InDefinition (const FInteractionDefinition)

SetupBuffersAndBindings

bool SetupBuffersAndBindings(const FInteractionDefinition &PolicyDefinition,
                             TSharedPtr<IModelInstanceRunSync> ModelInstance)

Sets up all buffers and bindings for the policy.

Parameters

  • PolicyDefinition (const FInteractionDefinition)

  • ModelInstance (TSharedPtr<IModelInstanceRunSync>)

Attributes

PolicyDefinition

FInteractionDefinition PolicyDefinition

Defines the observation and action spaces for this policy.

ModelData

TObjectPtr<UNNEModelData> ModelData

The neural network model data asset used for inference.

RuntimeName

FString RuntimeName

The runtime name for inference execution.

MaxStateSequenceLength

int MaxStateSequenceLength = 1

Maximum sequence length for state buffers with a sequence dimension.

bNetworkLoaded

bool bNetworkLoaded = false

Flag indicating whether the neural network model loaded successfully.

ActionBuffer

TInstancedStruct<FNNEPointBuffer> ActionBuffer

Buffer storing action data for neural network output.

ObservationBuffer

TInstancedStruct<FNNEPointBuffer> ObservationBuffer

Buffer storing observation data for neural network input.

StateBuffer

TArray<FNNEStateBuffer> StateBuffer

Array of buffers storing recurrent state for sequence-based models.

Source: Source/ScholaNNE/Public/Policies/NNEPolicy.h