Class UCameraSensor

Defined in File CameraSensor.h

Inheritance Relationships

Base Types

public USceneCaptureComponent2D
public IScholaSensor (Class IScholaSensor)

class UCameraSensor : public USceneCaptureComponent2D, public IScholaSensor

Source: Source/Schola/Interactors/Public/Sensors/CameraSensor.h Camera sensor that collects 2D image observations from the environment.

This sensor uses a SceneCaptureComponent2D and a RenderTarget to capture images from the environment. The captured images are converted to normalized floating-point arrays suitable for neural network input.

To use this sensor:

Create a UTextureRenderTarget2D asset in the Unreal Editor.
Assign it to the TextureTarget property (inherited from USceneCaptureComponent2D).
Configure which color channels to include via EnabledChannels.

Change the CaptureSource setting to choose the mode of operation:

SCS_FinalColorLDR: Standard RGB rendering
SCS_SceneDepth: Depth-only rendering
SCS_SceneColorSceneDepth: Combined RGB and depth

RenderTarget setting recommendations: Advanced –> Shared = true

SceneCaptureComponent2D setting recommendations: bCaptureEveryFrame = true bRenderInMainRenderer = true CompositeMode = overwrite

https://dev.epicgames.com/documentation/en-us/unreal-engine/1.7—scene-capture-2d

https://dev.epicgames.com/documentation/en-us/unreal-engine/BlueprintAPI/RenderTarget

Public Functions

Symbol	Details
`GenerateId`	Generate a unique ID string for this sensor.
`GetInvalidChannels`	Determine which channels are invalid for the current capture configuration.
`GetNumChannels`	Get the number of enabled and valid color channels.
`CollectObservations_Implementation`	Collect image observations from the render target.
`GetObservationSpace_Implementation`	Get the observation space for this sensor.
`InitSensor_Implementation`	Initialize the camera sensor.

GenerateId

FString GenerateId() const

Generate a unique ID string for this sensor.

Includes capture source, render target format, enabled channels, and resolution.

Returns:

FString describing the sensor configuration (e.g., “Camera_SCS_FinalColorLDR_RTF_RGBA8_RGB_W128_H128”)

Attributes: const

Source: Source/Schola/Interactors/Public/Sensors/CameraSensor.h

Implementation: Source/Schola/Interactors/Private/Sensors/CameraSensor.cpp

GetInvalidChannels

uint8 GetInvalidChannels() const

Determine which channels are invalid for the current capture configuration.

Some capture sources (e.g., depth) don’t support all RGBA channels. This method checks the CaptureSource and RenderTargetFormat to determine restrictions.

Returns:

uint8 Bitmask of invalid channels for the current configuration

Attributes: const

Source: Source/Schola/Interactors/Public/Sensors/CameraSensor.h

Implementation: Source/Schola/Interactors/Private/Sensors/CameraSensor.cpp

GetNumChannels

int GetNumChannels() const

Get the number of enabled and valid color channels.

Counts the channels that are both enabled and valid for the current capture configuration.

Returns:

int Number of channels that will be included in observations

Attributes: const

Source: Source/Schola/Interactors/Public/Sensors/CameraSensor.h

Implementation: Source/Schola/Interactors/Private/Sensors/CameraSensor.cpp

CollectObservations_Implementation

virtual void
CollectObservations_Implementation(FInstancedStruct &OutObservations) override

Collect image observations from the render target.

Reads pixels from the render target, extracts enabled channels, and normalizes to [0,1]. The resulting BoxPoint has shape [NumChannels, Width, Height] in CHW (channel-first) format.

Parameters

OutObservations – [out] A BoxPoint that will be populated with normalized pixel values

#	Direction	Name	Type	Description
1	—	`OutObservations`	`FInstancedStruct &`	A BoxPoint that will be populated with normalized pixel values

Attributes: virtual

Source: Source/Schola/Interactors/Public/Sensors/CameraSensor.h

Implementation: Source/Schola/Interactors/Private/Sensors/CameraSensor.cpp

GetObservationSpace_Implementation

virtual void GetObservationSpace_Implementation(
    FInstancedStruct &OutObservationSpace) const override

Get the observation space for this sensor.

Returns a BoxSpace describing the image dimensions and channels. Each dimension is bounded [0.0, 1.0] for normalized pixel values. Shape is [NumChannels, Height, Width] in CHW format.

Parameters

OutObservationSpace – [out] The observation space definition to be populated

#	Direction	Name	Type	Description
1	—	`OutObservationSpace`	`FInstancedStruct &`	The observation space definition to be populated

Attributes: const, virtual

Source: Source/Schola/Interactors/Public/Sensors/CameraSensor.h

Implementation: Source/Schola/Interactors/Private/Sensors/CameraSensor.cpp

InitSensor_Implementation

virtual void InitSensor_Implementation() override

Initialize the camera sensor.

Creates a default render target if none is assigned. Configures render target settings.

Attributes: virtual

Source: Source/Schola/Interactors/Public/Sensors/CameraSensor.h

Implementation: Source/Schola/Interactors/Private/Sensors/CameraSensor.cpp

Public Members

Symbol	Details
`EnabledChannels`	Bitmask of enabled color channels.

`EnabledChannels`

uint8 EnabledChannels = 15

Bitmask of enabled color channels.

Set bits determine which RGBA channels are included in observations.