Integrating VRS in The Riftbreaker

Originally posted May 13, 2022

Piotr Bomak (EXOR)

Andrzej Czajkowski (EXOR)

Paweł Lekki (EXOR)

Steffen Wiewel (AMD)

Hello everyone!

We are EXOR Studios – an independent game development studio based in Szczecin, Poland. Since 2007, we have produced a number of titles for PC and all major console platforms, using our own technology. Our latest project, The Riftbreaker, was released in October 2021 to players’ and critics’ acclaim.

The Riftbreaker Gameplay Trailer - Watch now!

(opens YouTube in a new window)

Thanks to our partnership with AMD, The Riftbreaker will offer a large number of cutting-edge technological solutions, such as raytraced shadows and ambient occlusion, as well as FidelityFX™ Contrast Adaptive Sharpening (CAS), FidelityFX Super Resolution 1.0 (FSR 1.0)^[1] and FidelityFX Variable Shading (VS) – which is also going to be the topic of this article.

AMD Features Showcase - Watch now!

(opens YouTube in a new window)

Introduction

Variable Rate Shading is an exciting new technique that allows the developer to utilize GPU cycles much more efficiently with the goal of reducing GPU work where it doesn’t contribute to the final frame. Modern gaming tends to push the screen resolution higher and higher. 4K displays are common, just like a variety of widescreen or multi-monitor setups. As display resolution goes up, the cost of rendering each frame of a game does too, so looking for novel ways to increase performance without significantly reducing image quality become key. The human visual system is great at picking out high fidelity detail, which means you can try and optimise areas that don’t have as much.

Variable Rate Shading aims to take advantage of this phenomenon by working under the assumption that it is not always necessary to compute every single pixel of a frame to get a decent result. For this assumption to work, it is crucial to ensure that the end-user will not see the difference between the original and the post-processed image. AMD FidelityFX Variable Shading provides an open source header implementation to easily integrate Variable Rate Shading into your game.

Requirements

To use the AMD FidelityFX Variable Shading library, a GPU must support D3D12_VARIABLE_SHADING_RATE_TIER_2 , which is a part of the DirectX® 12 Ultimate feature set. The library is open source and can be downloaded here.

The algorithm

FidelityFX VS is a compute shader that analyzes an information source image in real-time, classifying which areas of the image can be rendered at a lower sampling rate. The algorithm can be divided into a few basic steps:

Dividing an information source image into tiles.
Classification of image tiles based on pixel luminance variance.
Applying image tile classification feedback from additional sources (e.g. velocity buffer, UI mask). This step is optional.
Passing the resulting classification mask to the VRS DX12® API to render the new image frame.

The base information source image is the last rendered frame. The algorithm divides the image into small tiles and classifies which tiles of the image contain pixels that are similar enough to be rendered at a lower sampling rate without any visible negative impact on the image quality.

The algorithm compares pixels within each tile using a classification function. The result is the maximum luminance variation value of the pixels within the tile. The highest variance value within a tile is used to assign a proper shading rate for each image tile. The DirectX® 12 VRS API later uses the resulting texture as a look-up table to define the shading rate.

Image tile classification function

The classification algorithm used for comparing the neighboring pixels is crucial for the entire algorithm to work. The AMD FidelityFX Variable Shading library uses luminance as a basis for making that comparison. The function compares the luminance of pixels within each screen tile to determine the maximum variance value within each tile. If the variance is low enough, that a tile of pixels can be rendered at a lower sampling rate to maximize performance without noticeable loss in quality. It is up to the developer to decide what variance threshold is acceptable.

Since the input image is the last rendered frame, FidelityFX VS allows us to use the velocity buffer of the current frame to eliminate any artifacts that come as a result of camera movement. This allows for another potential optimization – an obligatory decrease of shading rate for groups of pixels in fast motion.

The AMD FidelityFX VS shader makes it possible for the developer to exchange the classification function for one that is better suited for their game. The open-source nature of the solution also allows for other changes. One such possibility is modifying the initial render targets and buffers to provide the algorithm with even more data to further optimize the comparison process and shading rate assignment.

The following video is a demonstration of the AMD FidelityFX Variable Shading running in real time. Green squares mark the areas of the screen where the shading rate has been reduced. Red squares mark those areas where the shading rate is preserved.

The Riftbreaker - VRS Active Zones Display - Watch now!

(opens YouTube in a new window)

Shading rate image

The results of the VRS calculations are stored in a resized render target that reflects the number of tiles the input data was divided into. The size of those tiles depends on the GPU vendor. Information about that is provided by DirectX® through the D3D12_FEATURE_DATA_D3D12_OPTIONS6 structure in the ShadingRateImageTileSize field.

The resulting texture is called “shading rate image.” Each pixel of that texture is a value that informs the GPU what shading rate it should apply to the corresponding tile. The shading rate is determined independently for pixels neighboring horizontally and vertically. The results of both tests are unified into one value. At the end of this process, each pixel is given one of the possible shading rate values: 1×1, 1×2, 2×1, or 2×2.

The next step is to feed the resulting shading texture to the DirectX® API. The ID3D12GraphicsCommandList5::RSSetShadingRateImage method instructs the GPU to use the provided shading rate image during the render target synthesis process. By reading the values from the texture, the GPU can generate the correct image based on the shading rate values in the texture.

Rendering pipeline integration

This graph shows the rendering pipeline of The Schmetterling Engine. The VRS shading pass takes the previous rendered frame as a reference and applies the shading rate change to particular rendering passes, marked red on the graph. VRS carries out calculations individually for each of them based on the reference shading rate image.

Results

In the case of The Riftbreaker, a base-building/action-RPG crossover with a top-down camera view, there are a lot of opportunities to save precious GPU cycles. While many dynamic elements on the screen, such as the fully destructible vegetation and hordes of enemy creatures, are not particularly well suited for VRS usage, there are also large patches of terrain, areas covered in shadows, or rock formations that can be rendered at a lower shading rate using VRS. Moreover, the game features a dynamic day and night cycle. As the surroundings get darker, a larger portion of scene detail becomes less prominent, making the image more straightforward for the algorithm to interpret, thus yielding more performance.

1220px crop (1:1) from 4K image

Riftbreaker VRS benchmark results

	Rasterization only	Ray-traced shadows and ambient occlusion
VRS off	202.13 FPS	94.55 FPS
VRS on	224.67 FPS	99.33 FPS
Performance improvement	Up to 11%	Up to 5%

The Riftbreaker – GPU Benchmark. Captured by AMD Labs using a AMD Radeon™ RX 6800 XT GPU, AMD Ryzen™ 3900X CPU, Windows 10, AMD Adrenalin driver 22.2.3. Performance will vary.

Riftbreaker VRS shading thresholds

Resolution	Variance cutoff
X <= 1080p	0.025
1080p < X <= 1440p	0.035
1440p < X	0.040

Future works

A significant advantage of working with technology such as the AMD FidelityFX Variable Shading is the ability to implement new functions that further expand the capabilities of the shader or adapt the existing features to suit your game better.

When it comes to further developing the FidelityFX Variable Shading integration into The Riftbreaker, one place that stands out is the part of the screen obscured by user interface. A large portion of the screen is not visible to the player due to the amount of information we need to relay to the player through the UI, yet it is still rendered at full quality. The information which parts of the screen are currently obscured could be relayed to VRS and used to reduce the sampling rate of those areas.

Another aspect of our rendering pipeline that could utilize the extra power offered by FidelityFX Variable Shading is raytracing. Utilizing the shading rate image to adjust the number of rays to be cast is a very promising direction for our tech. It will definitely be worth investigating further.

Conclusion

Thanks to the open-source nature of the library, FidelityFX Variable Shading was very straightforward to implement in our custom game engine. With the increase in GPU performance, coming at minimal quality cost for the end-user, the implementation is a clear benefit for the players. We are looking forward to further developments in this area and the potential additional applications of this technique.

Thanks to this, the time required by the GPU to render each frame is noticeably lowered. Even though the gain is the most substantial in night time scenarios, the realistic lighting model in The Riftbreaker allows the VRS technique to optimize a lot of areas on the screen even in scenes with complex geometry, such as the player’s base, or during highly dynamic combat scenarios, where the surrounding area is much less important than the actors themselves.

Problems

Applying VRS to some passes might lower image quality with no performance gain or decrease the quality below any acceptable level. Applying VRS to bloom, for example, resulted in generating square artifacts instead of the desired soft effect. This is why it’s essential to carefully consider which passes VRS should affect.

It is also worth noting that the shading rate threshold values should be adjusted for the screen resolution. The resolution significantly affects the detail level of the image fed to the algorithm in the first place. The more detailed the texture, the higher the threshold value should be. In the case of The Riftbreaker we decided on implementing three resolution ranges with different threshold values: resolutions lower than or equal to 1080p; higher than 1080p, but lower than 1440p; and higher than 1440p.

Download original 4K PNGs of the uncropped non-debug screenshot above.