Optimization and best practices

Thread group size

As discussed before, vertex transformation is likely to be the main compute workload of the mesh shader and therefore we recommend setting the thread groups size to match the maximum number of vertices in a meshlet. In case, that is not possible, try to at least divide the maximum number of vertices in a meshlet evenly across the threads. In the case of Direct3D12, the thread group size is limited to 128 threads, thus matching our recommended meshlet size of \hat{V}=128. In order to utilize the GPU’s parallel architecture, we recommend setting the thread group size to 128, thereby enabling a 1:1 mapping of threads and vertices. In case of a lower vertex limit, e.g., \hat{V}=64, we recommend only adjusting the thread group size if \hat{T} is below 128, as – in most cases – having a dedicated thread for exporting extra primitives is faster. Our experiments show that if mesh shader occupancy is heavily limited by pixel shader or other dispatches running in parallel, reducing thread group size to e.g., 64 can yield small performance uplifts.

Output array size

The output of a mesh shader is defined as arrays in the function signature of the mesh shader.

void MeshShader(...,
                out vertices VertexAttribute verts[128],
                out indices uint3 triangles[256])

The size of these arrays represents – from the perspective of the GPU – the worst case for vertex and primitive output. Thus, the geometry engine uses these limits to reserve output space in the shader export before launching a mesh shader thread group.
Therefore, these limits should be set as low as possible, i.e., in the case of pre-computed meshlet to \hat{V} and \hat{T}.

Writing to shared arrays

The vertices and indices arrays shown above are shared across the thread group and any thread can write to any element of the array. The mesh shader has to transfer the data written to these array to the shader export for the primitive assembler to assemble and rasterize the triangles. On RDNA™ graphics cards, the exp instruction is used to export a single primitive or vertex to the shader export. The order of primitives and vertices in the shader export is defined by the order of threads in the thread group, i.e., the i-th thread in the thread group will write to the i-th vertex or primitive in the shader export.

For vertices this means that the i-th thread has to export the i-th vertex, otherwise triangles would be assembled incorrectly. For primitive connectivity information, the Microsoft DirectX® 12 Mesh Shader specification imposes the following constraints on the rasterization order:

Triangles generated by any individual mesh shader thread group are always retired by the rasterizer in the order that thread group specified it’s primitive outputs.

Thus, the i-th thread also must export the i-th primitive in order to adhere to this constraint.
If the i-th thread did not produce the i-th vertex or primitive, i.e., another thread has written to verts[i] or triangles[i], vertex and/or primitive information must be exchanged between thread via the group shared memory. This increases latency and resource requirements of the mesh shader. If mesh shader writes verts[i] and triangles[i] from the i-th thread, then these values can be directly exported to the shader export without being passed through group shared memory.

As the maximum thread group size is limited to 128 in Direct3D12, but the maximum number of vertices and primitives can be up to 256, threads have to write multiple vertices or primitives in some cases. On graphics cards with the RDNA™ 2 architecture, threads are limited to a single vertex and primitive export. Thus the thread group is extended with additional, hidden threads to export the remaining vertices and primitives. Data is exchanged with these thread via group shared memory.
Graphics cards with the RDNA™ 3 architecture can specify a wave-wide offset to the exp instruction and are thus able to export the i-th and i + N-th vertex or primitive from the i-th thread. The offset N in this case is usually 128.

In summary, we recommend writing the i-th vertex and i-th primitive from i-th thread in the thread group. If vertex or primitive count exceeds the thread group size, e.g., in the above recommended \hat{V}=128 and \hat{T}=256 configuration, we recommend to write vertices and primitives with a thread-group-sided stride, e.g., the i-th thread writes triangles[i] and triangles[i + 128].
The code below shows an example mesh shader exporting up to 128 vertices and up to 256 primitives.

[numthreads(128, 1, 1)]
[outputtopology("triangle")]
void MeshShader(in uint groupId : SV_GroupID,
                in uint threadId : SV_GroupThreadID, 
                out vertices VertexAttribute verts[128],
                out indices uint3 triangles[256],
                out primitives PrimitiveAttribute prims[256])
{
    Meshlet meshlet = meshlets[groupId];

    SetMeshOutputCounts(meshlet.vertexCount, meshlet.primitiveCount);

    for (uint i = 0; i < 2; ++i) {
        const uint primitiveId = threadId + i * 128;

        if (primitiveId < meshlet.primitiveCount) {
            triangles[primitiveId] = LoadPrimitive(meshlet, primitiveId);
            prims[primitiveId] = LoadPrimitiveAttributes(meshlet, primitiveId);
        }
    }

    if (threadId < meshlet.vertexCount) {
        verts[threadId] = LoadVertex(meshlet, threadId);
    }
}

Compute budget and mesh shader occupancy

As discussed above, the geometry engine has to reserve output space in the shader export before launching a mesh shader thread group. Space in the shader export is finite and can thus limit the overall mesh shader occupancy to a GPU-wide maximum number of mesh shader thread groups. Shader export memory size is designed such that regular mesh shaders can reach the rasterizer triangle throughput. For simple mesh shaders, this means that a occupancy of ~25% can be enough to reach the triangle throughput limit.

More complex mesh shaders may see slightly higher occupancy, but also tend to be limited by available memory in the shader export. Thus, we recommend to only perform required computations in the mesh shader and pre-process any other data beforehand, e.g., vertex reuse information for primitives.

The compute-like programming model of mesh shaders also enables the use of group shared memory for communication between threads or waves. However, as group shared memory is limited and allocating larger amounts decrease overall (mesh-) shader occupancy, we recommend to not use group shared memory to temporarily store all vertices or primitives of a mesh shader thread group.

Mesh Shaders are typically compiled in wave64 mode, i.e., 64 threads per wave, which means only two waves are needed for a thread group size of 128 threads. Using certain instructions, e.g. WavePrefixSum, may cause the shader to be compiled in wave32 mode, thus requiring twice as many waves to execute the same thread group. This also causes mesh shader occupancy to appear higher without running more thread groups in parallel.

Occupancy of mesh shaders can also be temporarily limited by pixel shader thread groups if, for example, one or more meshlets cover enough screen area to fill the GPU with pixel shader operations. Pixel shaders are in this case prioritized over mesh shaders to relieve output pressure from the rasterizer. Mesh Shader occupancy (green) is limited by pixel shader workload (blue).

Primitive attributes

In addition to the per-vertex attribute data, mesh shaders can also specify per-primitive attributes. In the pixel shader, these attributes can be read like any other per-vertex attribute, with the main difference being that these attributes are not interpolated across the primitive.

struct PrimitiveAttribute {
    float4 primitiveColor : COLOR0;
}

[numthreads(128, 1, 1)]
[outputtopology("triangle")]
void MeshShader(...
                out primitives PrimitiveAttribute prims[256])
{
    prims[primitiveId] = LoadPrimitiveAttributes(meshlet, primitiveId);
}

Passing certain attributes as primitive attributes can reduce vertex duplication and/or help reduce memory traffic by reducing the overall size of the attribute memory written by the mesh shader. Some examples include flat-shading, whereby the normal attribute is specified per-primitive, rather than per-vertex or texture mapping systems like Ptex, which specify different texture coordinates for each primitive. Writing these attributes as primitive attributes allows the pixel shader to still read the correct attribute values without duplicating the vertex.

Further optimizations

• Mesh Shaders have access to group shared memory to communicate and share data across the thread group. Group shared memory should be used to share data and computation results which will be frequently used by multiple threads. We will show an example of this in an upcoming blog post on index buffer decompression.

• Use SV_CullPrimitive if you need to cull some triangles between other, visible triangles. The primitive assembler can load multiple triangles for assembly and discard any culled primitives with minimal overhead. These culled primitives will however still count towards the total primitive count of the mesh shader.

• Call SetMeshOutputCounts early on in the mesh shader. This call will reserve output memory for storing vertex and primitive attributes (other than the vertex position). Requesting memory early helps hiding the latency required for memory allocation in the geometry engine.

• Most if not all recommendations for compute shader optimization also apply to mesh shaders. This includes preferring aligned memory reads, loading data in a cache-friendly way, and avoiding dependent memory loads.

Work amplification

As described in the first blog post, mesh shaders have a compute-like programming model, but still need to be launched through the geometry engine in order to reserve output space for indices and vertex positions in the shader export. In addition to this requirement, we also want the amplified work to be executed across the entire GPU, thereby making use of the massive parallelism available on modern GPUs.
To enable a load-balanced launch of mesh shaders through the geometry engine, the DispatchMesh intrinsic calls are communicated back to the command processor and processed in much the same way as DispatchMesh commands from the CPU. To comply with the specified rasterization order, the command processor has to adhere to a strict order for processing these commands. Commands need to be executed in the same order as the amplification shader thread groups were launched.

This whole process comes at the expense of added latency to the rendering process. To recover this latency, a high enough amplification rate, i.e., the average number of mesh shader thread groups launched by a single amplification shader thread group is needed.

In scenarios in which the amplification shader performs per-meshlet culling operations, i.e., checking per-element visibility of a pre-determined set of meshlets, we recommend to process at least 32 or 64 elements, e.g., meshlets, per amplification shader thread group. By choosing the amplification shader thread group size accordingly, culling can be implemented using the WavePrefixCountBits and WaveActiveCountBits wave intrinsics on RDNA™ graphics cards.

bool visible = false;
const int meshletId = threadId;
if (meshletId < meshletCount)
{
    visible = IsMeshletVisible(Meshlets[meshletId]);
}

if (visible)
{
    const uint idx = WavePrefixCountBits(visible);
    payload.meshletIds[idx] = meshletId;
}

const uint meshletCount = WaveActiveCountBits(visible);
DispatchMesh(meshletCount, 1, 1, payload);

Increasing the number of elements processed by a single amplification shader thread group reduces the number of commands sent back to the command processor and thereby reduces latency.

For other applications, such as dynamic level-of-detail selection or geometry amplification (e.g., subdivision surfaces), a similar principle applies. A single amplification shader thread group should typically launch at least 32 mesh shader thread groups to optimally hide the added latency.

Payload

An amplification shader thread group can further output a payload of up to 16k bytes. All mesh shader thread groups launched by that amplification shader thread group have access this 16k payload. This payload needs to be stored in memory in order for the mesh shaders to read it and thus needs to be written to memory by the amplification shader. Larger payloads can thus be quite costly in terms of memory usage and memory read/write operations. We recommend to keep the amplification shader payload size to a minimum and only transfer data that cannot be directly loaded, computed, or inferred by the mesh shader.

A typical amplification shader payload contains an array of parameters, with each element being processed by one or more of the subsequently launched mesh shader thread groups. If each element of the payload is only written by one thread, the compiler can optimize writes to the payload and directly write the element to memory.