Skip to content

Setting Up Inference

Setting Up Inference

This guide will explain how to use your trained RL agents in inference mode (i.e. without connecting to Python).

Convert a Checkpoint to Onnx

If you did not export to Onnx during training you will need to convert a checkpoint to Onnx. You can use the following scripts to create an Onnx model from your checkpoint:

Terminal window
schola sb3 export --policy-checkpoint-path <CHECKPOINT_PATH> --output-path <ONNX_PATH>

These commands will create an Onnx model in a standardized format compatible with Schola that can be used in the next section.

Load an Onnx Model into Unreal Engine

Once you have your Onnx model you can import it into Unreal Engine by dragging and dropping the .onnx file into the content browser. This will create a new Onnx model data asset in your project.

Setting up Your Unreal Engine Level

Schola’s inference system consists of three main components:

  1. Agent - Any object implementing the IAgent interface that defines observation and action spaces
  2. Policy - A UNNEPolicy that loads your trained ONNX model and performs inference
  3. Stepper - A USimpleStepper (or UPipelinedStepper) that coordinates the observation-inference-action loop

Follow these steps to set up inference in your project:

Step 1: Implement the IAgent Interface

Create a class (Actor, Component, or any UObject) that implements the IAgent interface. You must implement these methods:

  • Define() - Specify the observation and action spaces for your agent

  • Observe() - Collect current observations from the environment

  • Act() - Execute actions provided by the policy

  • GetStatus() / SetStatus() - Manage agent state

Create a Blueprint class and add the Agent interface. Implement the Define, Observe, and Act events.

Step 2: Create and Configure the Policy

Create a UNNEPolicy object and configure it with your ONNX model:

  1. In your Blueprint or C++, create a UNNEPolicy object
  2. Set the Model Data property to the ONNX model data asset you imported
  3. Set the Runtime Name to your desired inference runtime (e.g., “NNERuntimeORTCpu” or “NNERuntimeORTDml”)
  4. Call Init() with the agent’s interaction definition

  • Add a UNNEPolicy variable to your Blueprint

  • In BeginPlay, call Define on your agent to get the interaction definition

  • Call Init on the policy, passing the interaction definition

  • Set the Model Data and Runtime Name properties in the details panel

Step 3: Create and Initialize the Stepper

Create a USimpleStepper to manage the observation-inference-action loop:

  1. Create a USimpleStepper object
  2. Call Init() with your agent(s) and policy
  3. Call Step() each frame (e.g., in Tick()) to run inference

  • Add a USimpleStepper variable to your Blueprint

  • In BeginPlay, call Init with an array of agents and your policy

  • In Tick, call Step on the stepper