For advanced users only
Much like Direct3D 12 was designed to stand alongside Direct3D 11, amplification and meshlet shaders were introduced to stand alongside the conventional geometry processing pipeline and their flexibility results from increased developer responsibility. If you have "nothing interesting" to express with them, it is highly likely you will incur performance penalties trying to badly reinvent the old geometry pipe. This is not a free lunch or a silver bullet.
GPUs have come a long way since their introduction as "3D" graphics accelerators via highly-specialized and largely fixed-function hardware. These days, they're far more general and infinitely more programmable which makes them applicable to a wide array of computing problems. Hardware companies are especially keen on this emerging market and are consciously trying to distance themselves from the term Graphics Processing Unit in favor of Compute Accelerator. The jaded among you might attribute this to a marketing ploy, but there is actual merit it. While GPUs still have some fixed function hardware within their dedicated engine for graphics acceleration, they're largely generalized and massively-parallel compute powerhouses. In fact, the majority of the conventional graphics pipeline is implemented on the back of thousands of compute cores. We're almost 15 years away from the last time we spoke of discrete vertex and pixel pipelines.
Graphics APIs and the underlying "pipeline" have, unfortunately, failed to keep in lockstep with the evolution of GPUs, mostly out of comfortable complacency because the classic IMR graphics pipeline is, more or less, a success story which delivers great performance for everything but the most demanding of applications. To push boundaries beyond what's comfortable, Independent Hardware Vendors, Independent Software Vendors and platform holders have come together to enable next-generation graphics APIs such as Direct3D 12 and Vulkan in the mid 2010s. Even with all these advances, a big part of pushing more detail on screen remained in limbo, restructuring the geometry processing pipeline. For the past few years, this has been the hot topic among realtime computer graphics researchers.
The fruits of said labor have been available for two years at least, but adoption in the mainstream will be a tenuous one. This is especially true of personal computers where graphics hardware prices have been skyrocketing due to Nvidia's de facto monopoly. Today, I am going to give you a tour of what you can expect on the PC and, of course, Xbox Series X which will come with full support out of the (x)box. The same is true of PlayStation 5, which should help early adoption because AAA developers use consoles as a baseline and PCs as port targets.
Why change?
The coarse diagram found below this paragraph should be old hat to you by now. It is a high level overview of a basic IMR architecture with one of its crown jewels, the ubiquitous vertex shader. For simplicity, it's missing tessellation and geometry shading stages. We'll get to the former in due time while the latter should be bleached out of your mind.
A vertex shader, at its core, is nothing more than a wrapped compute kernel with special semantics and output requirements, executing on many cores. What predicates thread dispatch and data shoveling is, in D3D parlance, the input assembler stage.
Hardware companies have their own designations for the piece of fixed-function hardware associated with this stage, we will adopt Nvidia's terminology for the duration of this article. On Nvidia Pascal GPUs and below (Maxwell, Kepler, Fermi...) it is known as the Primitive Distributor which is a fairly apt name given that it takes the inherently serial index buffer and determines the best way to cut up (distribute) the supplied vertex buffer into batches which maximize utilization and on-die memory sharing once thread dispatch begins.
Once vertex processing ("shading") is finished, the next piece of fixed-function hardware hit before the rasterizer is the viewport clipping (and culling) stage. It takes the processed vertices and, as the name implies, performs fast clipping and culling operations against the configured viewport. While it is very good at what it does, it is preferable to perform culling before processing vertices. Like with the primitive distributor, fixed-function hardware needs to step down a bit in the interest of flexibility.
How the Primitive Distributor hurts you
First thing I need to impress upon you is that index buffers exist to avoid vertex duplication by generating index duplication. Depending on representation, indices can be as wide as \(32\) bits. Therefore, this is extremely wasteful during processing. To illustrate what the primitive distributor does, let index buffer \(\mathbf{IB}\) for primitive_topology::triangle_list contain the sequence of indices [7123, 7124, 7125, 7123, 7124, 7126, ...]. From here, we need to isolate the unique vertices referenced into a batch \(\mathbf{B_v}\) as [7123, 7124, 7125, 7126, ...], bear in mind we use the original indices to refer to the vertices for readability. Original because for every such vertex batch, a special primitive indexing batch is generated relative to it, allowing indices to be represented with significantly fewer bits (such as $8$ bits), \(\mathbf{B_i}\) as \([0, 1, 2, 0, 1, 3 \space ...]\).
Given that thread dispatch granularity on Nvidia GPUs is 32 threads (the size of a warp), to maximize utilization and performance, the primitive distributor will try to fill up as many 32-wide vertex and primitive batches as possible. While this gives you a clue on why processing tiny vertex buffers is quite expensive on a large scale, the real performance culprit is the fact that said index deduplication is performed every frame, dozens and even hundreds of times a second. That's bad, especially in the case of static geometry.
What are we to do?
Further embrace the compute accelerator nature of modern GPUs. Modify the hardware in such a way that it doesn't hinge on the Primitive Distributor and the misplaced viewport clipping/culing stage. The classic graphics pipeline can be implemented in terms of the more flexible system which doesn't have a set number of stages and it can be done by the runtime and the driver for backwards compatibility for the many people who can't beat the fixed function hardware on their own.
Don't do unnecessary work. Primitive Distributor's work can be largely kicked off to a build/bake step in the engine. Fundamentally, the Primitive Distributor will break up a mesh every time into a bunch of meshlets which are going to run in waves. The Primitive Distributor is also not magic, if your topology is trash, performance will suffer. Once it's relegated to a bake step, a lot more time can be spent on optimizing the meshlets for online processing, shortening the path to the rasterizer at runtime.
Arbitrary geometry representations follow naturally as you no longer have a requirement of providing conventional vertex and index buffers, allowing for novel and compact representations.
Amplification & meshlet shaders
As discussed in the last section, threads are dispatched on a hardware-specific granularity within which it is possible to share data and perform otherwise unavailable performance-enhancing operations known as wave intrinsics. Thus it's necessary to optimally break up a mesh into cache-friendly meshlets at bake time (as opposed to runtime). Furthermore, issuing thread groups sequentially, once per-vertex followed by an optional per-primitive stage is not very flexible. It is preferable to intermittently operate on both per-primitive and per-vertex data within each thread of execution in order to maximize opportunities to avoid work. As the old addage goes, the most performant operation will always be a NOP.
(De)amplified meshlet killed the vertex star
One thing that made vertex shaders such a success story was the tremendously simple programming model. Vertex comes in, processed vertex comes out. Geometry shaders are similarly simple as they perform work on a per-primitive basis. While they were a good-ish idea on paper, executing threads on a per-primitive basis and doing so after the fact is a somewhat limiting factor... Doubly so because anything a geometry shader can do, a compute shader can do better. Thus, it made sense to offer an alternative to vertex and geometry shader combos in the form of the novel mesh(let) shader. To account for tessellation, culling and instancing concerns, an aptly-named amplification shader precedes the meshlet shader which potentially amplifies or "dampens" input geometry while allowing early removal of offscreen surfaces before any expensive per-primitive and per-vertex processing takes place. In Nvidia parlance, it is also known as the task shader.
Put simply, the amplification shader determines just how many mesh(let) shaders will have to be executed. Remember how I told you that most of the graphics pipeline is basically a bunch of wrapped compute shaders with intermittent fixed-function stages? You can see it here clearly as now you have access to shared memory which means the usual thread-addressing IDs are fed into the shader (in HLSL, these would be sysvals such as SV_GroupThreadID and SV_GroupID). As the meshlet shader needs to produce some useful output, you have to write to two arrays, one for per-vertex data and the other for per-primitive data. There is flexibility in how much data you write but upper bounds must be specified ahead of time.
Specific limits under D3D12 Ultimate
Meshlet shaders under feature level 12_2 (DX12 Ultimate) require that the number of threads in a thread group is at most 128. The number of vertices and primitives you output is limited to at most 256 while vertex data itself with all the attributes should not exceed 32 KiB. There are also some limits on the amount of group shared memory, but the initial offering is fairly generous (at around 28 KiB).
For Nvidia hardware, it is recommended that you don't exceed 64 vertices output and 126 primitives. This is still way up from the original 32 on both counts. 126 being an "odd" number is thanks to allocation granularity (128 bytes) for primitive indices on Turing GPUs (the first generation). At 127, you're going into trigger a subsequent allocation of 128 bytes.
Just like D3D12 offered the developer more control over the GPU by having the runtime and driver step aside for you to make your own trade-offs, tessellation in the conventional sense is gone in this new model. This is good because the fixed-function tessellator has a predefined and immutable pattern used to generate more detail based on the hull shader's constrained output. Nowadays, for better or worse, both the inputs and outputs for tessellation are user-defined.
At this point, I am hoping it is becoming clear to you that this added flexibility may very well mean a performance catastrophe if you don't know what you're doing. Beating old fixed-function hardware is only possible if you know what you want and are willing to go above and beyond when it comes to availing yourself of the new programming model, especially taking advantage of the ability to perform culling ahead of time. If else, don't use this. If you are thinking of redoing the classic geometry pipeline in terms of amplification and meshlet shaders, you're missing the point.
The nitty-gritty details
Amplification and meshlet shaders are easier to define in terms of their limits than their possibilities. It should be very clear to you by now that there is no catch-all code snippet I can give you to put into your application. With that caveat in mind, here's a simple, non-production, introductory meshlet shader in HLSL for you to get a feel for things:
// I hate CamelCasing.
struct ThreadInformation {
uint3 ID : SV_DispatchThreadID;
uint3 GroupRelativeID : SV_GroupThreadID;
uint3 GroupID : SV_GroupID;
uint LinearGroupID : SV_GroupIndex;
};
[numthreads(64, 1, 1)] [outputtopology("triangle")]
void processMeshlet(in ThreadInformation threadInformation, out vertices Vertex outputVertices[64], out indices uint3 outputIndices[126]) {
// Structured buffer, probably
Meshlet meshlet = meshlets[threadInformation.belongingGroupID.x];
SetMeshOutputCounts(meshlet.vertexCount, meshlet.primitiveCount);
uint linearGroupRelativeID = threadInformation.LinearGroupID;
// Per-vertex processing
if(linearGroupRelativeID < meshlet.vertexCount) {
// Get the associated vertex index and read from yet another structured buffer
Vertex vertex = vertices[meshlet.vertexOffset + linearGroupThreadID];
// ... Do your vertex processing.
}
// Per-primitive processing
if(linearGroupRelativeID < meshlet.primitiveCount) {
// Similarly, process and assemble primitives, store result into outputIndices.
}
}
You will need to upgrade your compiler to get this to work, FXC has now been surpassed by DXC. While HLSL is still a soaring dumpster fire, it's actionable. Can't hold a candle to the C++14-based Metal Shading Language, though. However, with the advent of SPIR-V and going from DXBC to DXIL (thanks, Clang), you can produce your own shading language that does exactly what you want it to. Note that using FXC generated DXBC will incur load-time conversions to DXIL behind the scenes.
How do amplification and meshlet shaders "talk"?
In in DX12 Ultimate / HLSL parlance, we have payloads which are at most 16 KiB worth of data per triggering meshlet shader and is used instead of explicit structured buffers in the example above. Instead, those are moved to the amplification shader where input decoding of your custom geometry format is performed followed by culling computations of arbitrary complexity, tradeoffs are yours to make. A skeleton looks like this:
struct Payload {
// User-defined and arbitrary, sky is the limit (16 KiB)
};
[numthreads(64, 1, 1)]
void prepareMeshletDispatch(ThreadInformation threadInformation) {
Payload meshletPayload;
// Decode your geometry format, cull, prepare tessellation and instancing.
// Then dispatch a fresh meshlet for processing:
DispatchMesh(numberOfThreadGroupsToDispatch, 1, 1, meshletPayload);
}
[numthreads(64, 1, 1)] [outputtopology("triangle")]
void processMeshlet(in ThreadInformation threadInformation, in payload Payload meshletPayload,
out vertices Vertex outputVertices[64], out indices uint3 outputIndices[126]) {
// Use whatever your payload is to generate work for the rasterizer
}
Again, how you write these two will highly depend on what kind of application you have. Careful thought and testing will be required to determine whether you benefit from this new approach at all. Don't pull a Deus Ex: Mankind Divided where the D3D12 implementation actually had worse performance than the D3D11 one.
How do I access this from the API?
Once this hits the API, a new method on the graphics command list interface will be available:
void ID3D12GraphicsCommandList6::DispatchMesh(UINT ThreadGroupCountX, UINT ThreadGroupCountY, UINT ThreadGroupCountZ);
Similarly, indirect meshlet dispatching will be supported via the upcoming D3D12_INDIRECT_ARGUMENT _TYPE_DISPATCH_MESH enumeration and D3D12_DISPATCH _MESH_ARGUMENTS structured member. As you can see, the majority of the big brain time will be sequestered inside of your geometry encoding scheme, meshlet preparation and the amplification and meshlet shaders.
What about rasterization ordering?
In the unlikely case of just meshlet shaders, total ordering is guaranteed which is built around the notion that you will be mostly dispatching 1D threadgroups. For each "primitive depth slice", every "primitive row" is sequenced top-to-bottom one after the other. Primitives within a row are fully ordered, low to high. Once you get to using amplification shaders which spawn threadgroups on their own, like every other algorithm which depends on effective indirect dispatch, total ordering is out the window. Partial ordering is available on the basis of the just discussed internal total ordering of every meshlet shader dispatched. Due to the complexity of amplification shader work and provisions for indirect dispatch, stronger guarantees cannot and don't need to be made.
Once beyond the rasterizer, old rules apply. If you need specific ordering imposed for the purposes of your algorithms inside of the pixel shader, your only recourse is using Raster Order Views ("PixelSync" hardware needs to be present).
Before I go, primitive attributes...
With amplification and meshlets shaders, it is now possible to define per-primitive attributes without actually hacking around with stuff like provoking vertices and the rain that brings down on your parade. Programmatically, it's trivial to add to your meshlet shaders as an out argument:
out primitives PerPrimitiveData outputPrimitives[PRIMITIVE_COUNT_MAX]
All there is to it, PerPrimitiveData is just an arbitrary, user-defined struct. On it, you can put either various system values pertinent to primitive processing or something you want to add as part of your primitive tagging scheme.
And with that, I'm out. I'll do a deep dive once this is out for the general public, you can hit me up on Discord to discuss details, best practices and approaches which make good use of this programming model. As always, have fun and don't melt your GPUs.