Skip to content

UCameraSensor

Camera sensor that collects 2D image observations from the environment.

This sensor uses a SceneCaptureComponent2D and a RenderTarget to capture images from the environment. The captured images are converted to normalized floating-point arrays suitable for neural network input.

To use this sensor:Create a UTextureRenderTarget2D asset in the Unreal Editor Assign it to the TextureTarget property (inherited from USceneCaptureComponent2D) Configure which color channels to include via EnabledChannels

Change the CaptureSource setting to choose the mode of operation:SCS_FinalColorLDR: Standard RGB rendering SCS_SceneDepth: Depth-only rendering SCS_SceneColorSceneDepth: Combined RGB and depth

RenderTarget setting recommendations: Advanced -> Shared = true

SceneCaptureComponent2D setting recommendations: bCaptureEveryFrame = true bRenderInMainRenderer = true CompositeMode = overwrite

See also https://dev.epicgames.com/documentation/en-us/unreal-engine/1.7—scene-capture-2d See also https://dev.epicgames.com/documentation/en-us/unreal-engine/BlueprintAPI/RenderTarget

class UCameraSensor : public USceneCaptureComponent2D, public IScholaSensor

Methods

GenerateId

FString GenerateId() const

Generate a unique ID string for this sensor.

GetInvalidChannels

uint8 GetInvalidChannels() const

Determine which channels are invalid for the current capture configuration.

GetNumChannels

int GetNumChannels() const

Get the number of enabled and valid color channels.

CollectObservations_Implementation

virtual void
CollectObservations_Implementation(FInstancedStruct &OutObservations) override

Collect image observations from the render target.

Parameters

  • OutObservations (FInstancedStruct)

GetObservationSpace_Implementation

virtual void GetObservationSpace_Implementation(
    FInstancedStruct &OutObservationSpace) const override

Get the observation space for this sensor.

Parameters

  • OutObservationSpace (FInstancedStruct)

InitSensor_Implementation

virtual void InitSensor_Implementation() override

Initialize the camera sensor.

Attributes

EnabledChannels

uint8 EnabledChannels = 15

Bitmask of enabled color channels.

Source: Source/ScholaInteractors/Public/Sensors/CameraSensor.h