Static vs Dynamic Environments
Schola provides two types of environments for reinforcement learning: Static and Dynamic. These environments differ in how agents and trainers are managed, with the Static Environment
Static Environments
Static environments are designed for scenarios where the agents and trainers are predefined and do not change dynamically during the simulation. These environments are more rigid and are suitable for use cases where the structure of the environment remains constant. To create a static environment, you can use the AStaticScholaEnvironment class, which is a subclass of AbstractScholaEnvironment.
Key Differences
-
Agent Registration: Agents are registered using pointers to AActor’s controlled by
AbstractTrainer. This method is implemented by child classes.
Dynamic Environments
Dynamic environments are designed for scenarios where agents and trainers can be created, modified, or removed during the simulation. These environments are more flexible and are suitable for use cases where the structure of the environment evolves over time. An example would be a zombie game where a variable number of zombies are spawned throughout the game. To create a dynamic environment, you can use the DynamicScholaEnvironment class, which is a subclass of AbstractScholaEnvironment.
Key Differences
-
Agent Registration: Agents are registered using a pair of UClass Objects, one a child of
AbstractTrainerand the other a child of Pawn (Object Reference). This method can be overridden in Blueprints. -
Agent Spawning: Agents can be spawned using the
SpawnAgent()method. This allows for more complex and adaptive environments where agents can be added or removed based on the simulation’s needs.
