Lightweight attention-based indirect illumination

Real-time global illumination (GI) means balancing fidelity against tight compute budgets. Neural GI methods split into two camps: screen-space approaches are fast but miss off-screen light, while scene-wide approaches recover it through data-gathering that doesn’t fit standard rendering pipelines. At SIGGRAPH 2026 we presented a lightweight, attention-based model that bridges the gap by shifting the budget from model complexity to richer inputs — augmenting screen-space G-buffers with reflective shadow map (RSM) ¹ data rendered from each light source. This captures geometry beyond the camera’s view, improving indirect shadow quality and, unlike prior work ², generalizing to scenes not seen during training.

Our approach

We predict only the indirect illumination component. The key signal is reflected flux buffers rendered from light-source perspectives, whose texels are treated as indirect virtual point lights (iVPLs); because they often capture geometry outside the current view frustum, they let the model reconstruct off-screen effects. A low-cost but noisy one-bounce (1spp) buffer from the main camera adds a further hint. We train our model on synthetic data rendered from 11 scenes.

Figure 1. Overview of our approach. Main-camera buffers (direct illumination, 1spp illumination, G-buffers, scene size) and RSM-camera buffers (G-buffers and reflected flux) are encoded, related by a multi-head attention module, and decoded into diffuse and specular indirect illumination that are composed with direct illumination.

Exploiting the linearity of light transport, we predict diffuse and specular indirect illumination separately and compose the final image. Three encoders — an iVPL MLP, a pixel-light UNet, and a pixel-geometry MLP — feed a multi-head attention module in which pixel-geometry embeddings are the query and the combined light data provides keys and values. Two MLP decoders then produce the diffuse and specular outputs. The UNet is used for the pixel-light encoder specifically to aggregate spatial context from the noisy one-bounce buffer.

Figure 2. Result from an unseen test-only scene. We compare our final global illumination (c) and indirect illumination (g) with BCNN ³ (b, f) and the ground truth (d, h). Arrows highlight areas where our method predicts plausible indirect illumination.

Results

On a test-only scene disjoint from training, our method accurately reconstructs complex specular highlights on the couch, armchair, and teapot, despite little useful signal in the noisy one-bounce guide (Figure 2). It produces noticeably richer indirect illumination than a CNN-based indirect illumination model (BCNN) trained on the same data. A residual color shift remains, but adding more training scenes consistently reduces it — suggesting the model is not yet capacity-constrained. The network is lightweight (2.19M parameters, 432 GFLOPs) and, without inference optimizations (FP32 PyTorch model), runs in 45.56 ms at 512×512 on an AMD Instinct™ MI250 Accelerator ⁴.

Figure 3. Video of our results compared to a prior method and reference ground truth.

• Further details can be found in our poster presented at SIGGRAPH 2026.
• Discuss this research on the AMD Developer Community.
• Sign up to our AMD Developer Newsletter for the latest updates.

Disclaimers and attributions

Links to third party sites are provided for convenience and unless explicitly stated, AMD is not responsible for the contents of such linked sites and no endorsement is implied. GD-97.

PyTorch, the PyTorch logo and any related marks are trademarks of The Linux Foundation.

References and footnotes