How it started

We first started talking about mesh shaders in Mesa about two years (maybe more?) ago. At the time the only publicly available Vulkan mesh shading API was the vendor-specific NV_mesh_shader made by NVidia.

• At the time nobody quite fully understood what mesh shaders are supposed to be and how they could work on the HW. I initially anticipated that they could be made to work even on RDNA1, but this turned out false due to some HW limitations.
• It was unclear what was needed to get good performance.
• The NVidia extension was a good start, but it was full of things that don’t make any real sense on any other HW.
• Nobody had a clue how we would implement task shaders.

I was working together with Christoph Kubisch (from NVidia) who helped understand what this topic is all about. Caio Oliveira and Marcin Slusarz (from Intel, working on ANV) joined the adventure too.

How we made it work

We made the decision to start working on some preliminary support for the NV extension to get some experience with the topic and learn how it’s supposed to work. Once we were satisfied with that, we made the jump to the EXT.

NIR and SPIR-V

These are the common Mesa pieces that all drivers can use. The front-end and middleware code for mesh/task shader support are here. Caio created the initial pieces and I expanded on that as I found more and more things that needed adjustment in NIR, added a new storage class for task payloads etc. Marcin also chimed in with fixes and improvements.

AMD and Intel hardware work differently, so most of the hard work couldn’t be shared and needed to be implemented in the backends. However, some of the commonalities are implemented in eg. nir_lower_task_shader for the work that needs to happen in both RADV and ANV. There were dozens of merge requests that added support for various features, cleaned up old features to make them not crash on mesh shaders, etc. The latest is this MR which adds all the remaining puzzle pieces for the EXT.

Lowering the shaders in the backend

Because mesh shaders use NGG, the heavy lifting is done in ac_nir_lower_ngg which is already responsible for other NGG shaders (VS, TES, GS). The lowering basically “translates” the API shader to work like the HW expects. Without going into much detail, it essentially wraps the application shader and replaces some pieces of it to make them understandable by the HW.

There is now also an ac_nir_lower_taskmesh_io_to_mem for translating task payload I/O and EmitMeshTasksEXT to something the HW can understand.

Mesh shading draw calls in RADV

Previously I was mostly working on the compiler stack (NIR and ACO) and had little experience with the hardcore driver code, such as how draws/dispatches work and how the driver submits cmd buffers to the GPU. As I had near-zero knowledge of these areas, I had to learn about them, mostly by just reading the code.

So, I split the RADV work in two parts:

1. Mesh shader only pipelines. These required only moderate changes to radv_cmd_buffer to add some new draw calls, and minor work in radv_pipeline to figure out per-primitive output attributes and get the register programming right.
2. Task shader support. Due to how task shaders work on AMD HW, these required very severe refactoring in radv_device because we now have to submit to multiple queues at once. This is actually not finished yet because “gang submit” is still missing. Additionally, radv_cmd_buffer needed heavy changes to support an internal compute cmdbuf.

Naturally, during this work I also hit several RADV bugs in pre-existing use cases which just nobody noticed yet. Some of these were issues with secondary command buffers and conditional rendering on compute-only queues. There was also a nasty firmware bug, and other exciting stuff.

Let’s just say that my GPU loved to hang out with me.

Mesh shading on Intel

The Intel ANV compiler backend and driver implementation were done by Caio and Marcin and I just want to take this opportunity to say that I really enjoyed working together with them.

The guy who wrote the most mesh shaders on Earth

All of the above would have been impossible if I didn’t have some solid test cases which I could throw at my implementation. Ricardo Garcia developed the CTS (Vulkan Conformance Test Suite) testcases for both NV_mesh_shader and EXT_mesh_shader. During that work, Ricardo wrote several thousand mesh and task shaders for us to test with. Ricardo if you’re reading this, THANK YOU!!!

Conclusion

Implementing VK_EXT_mesh_shader gave me a very good learning experience and helped me get an understanding of parts of the driver that I had never looked at before.

What happens to NV_mesh_shader now?

We never wanted to officially support it, it was just a stepping stone for us to help start working on the EXT. The NV extension support will be removed soon.

When is it coming to my Steam Deck / Linux computer?

The RADV and ANV support will be included once the system is updated to Mesa 22.3, though we may be convinced to bring it to the Deck sooner if somebody finds a game that uses mesh shaders.

For NVidia proprietary driver users, EXT_mesh_shader is already included in the latest beta drivers.

Waiting for the gang…

We marked mesh/task shader support “experimental” in RADV because it has one main caveat that we are unable to solve without kernel support. Thanks to how we need to submit to two different queues at the same time, this can deadlock your GPU if you are running two (or more) processes which use task shaders at the same time. To properly solve it we need the “gang submit” feature in the kernel which prevents such deadlocks.

Unfortunately “gang submit” is not upstream yet. Cross your fingers and let’s hope it’ll be included in Linux 6.1.

Until then, you can use the RADV_PERFTEST=ext_ms environment variable to play your favourite mesh shader games!

Old stuff: the legacy geometry pipeline on GCN

Let’s start by briefly going over how the old GCN geometry pipeline works, so that we can compare old and new.

GCN GPUs have 5 programmable hardware shader stages for vertex/geometry processing: LS, HS, ES, GS, VS. These HW stages don’t exactly map to the software stages that are advertised in the API. Instead, it is the responsibility of the driver to select which HW stages need to be used for a given pipeline. We have a table if you want details.

The rasterizer can only consume the output from HW VS, so the last SW stage in your pipeline must be compiled to HW VS. This is trivial for VS and TES (yes, tess eval shaders are “VS” to the hardware), but GS is complicated. GS outputs are written to memory. The driver has no choice but to compile a separate shader that runs as HW VS, and copies this memory to the rasterizer. We call this the “GS copy shader” in Mesa. (It is not part of the API pipeline but required to make GS work.)

Notes:

• All of these HW stages (except GS, of course) use a model in which 1 shader invocation (SIMD lane, or simply thread in D3D jargon) corresponds to 1 output vertex. These stages (except GS) are also not aware of output primitives.
• Vega introduced “merged shaders” (LS+HS, and ES+GS) but did not fundamentally change the above model.

New stuff: NGG pipeline on RDNA

The next-generation geometry pipeline vastly simplifies how the hardware works. (At the expense of some increased driver complexity.)

There are now only 2 HW shader stages for vertex/geometry processing:

• Surface shader which is a pre-tessellation stage and is equivalent to what LS + HS was in the old HW.
• Primitive shader which can feed the rasterizer and replaces all of the old ES + GS + VS stages.

The surface shader is not too interesting, it runs the merged SW VS + TCS when tessellation is enabled. I’m not aware of any changes to how this works compared to old HW.

The interesting part, and subject of much discussion on the internet is the primitive shader. In some hardware documentation and register header files the new stage is referred to as simply “GS”, because AMD essentially took what the GS stage could already do and added the ability for it to directly feed the rasterizer using exp (export) instructions. Don’t confuse this with SW GS. It supports a superset of the functionality that you can do in software VS, TES, GS and MS. In very loose terms you can think about it as a “mesh-like” stage which all these software stages can be compiled to.

Compared to the old HW VS, a primitive shader has these new features:

• Compute-like: they are running in workgroups, and have full support for features such as workgroup ID, subgroup count, local invocation index, etc.
• Aware of both input primitives and vertices: there are registers which contain information about the input primitive topology and the overall number of vertices/primitives (similar to GS).
• They have to export not only vertex output attributes (positions and parameters), but also the primitive topology, ie. which primitive (eg. triangle) contains which vertices and in what order. Instead of processing vertices in a fixed topology, it is up to the shader to create as many vertices and primitives as the application wants.
• Each shader invocation can create up to 1 vertex and up to 1 primitive.
• Before outputting any vertex or primitive, a workgroup has to tell how many it will output, using s_sendmsg(gs_alloc_req) which ensures that the necessary amount of space in the parameter cache is allocated for them.
• On RDNA2, per-primitive output params are also supported.

How is shader compilation different?

Software VS and TES:
Compared to the legacy pipeline, the compiled shaders must now not only export vertex output attributes for a fixed number of vertices, but instead they create vertices/primitives (after declaring how many they will create). This is trivial to implement because all they have to do is read the registers that contain the input primitive topology and then export the exact same topology.

Software GS:
As noted above, each NGG shader invocation can only create up to 1 vertex + up to 1 primitive. This mismatches the programming model of SW GS and makes it difficult to implement. In a nutshell, for SW GS the hardware launches a large enough workgroup to fit every possible output vertex. This results in poor HW utilization (most of those threads just sit there doing nothing while the GS threads do the work), but there is not much we can do about that.

Mesh shaders:
The new pipeline enables us to support mesh shaders, which was simply impossible on the legacy pipeline, due to how the programming model entirely mismatches anything the old hardware could do.

How does any of this make my games go faster?

We did some benchmarks when we switched RADV and ACO to use the new pipeline. We found no significant perf changes. At all. Considering all the hype we heard about NGG at the hardware launch, I was quite surprised.

However, after I set the hype aside, it was quite self-explanatory. When we switched to NGG, we still compiled our shaders mostly the same way as before, so even though we used the new geometry pipeline, we didn’t do anything to take advantage of its new capabilities.

The actual perf improvement came after I also implemented shader-based culling.

What is NGG shader culling?

The NGG pipeline makes it possible for shaders to know about input primitives and create an arbitrary topology of output primitives. Even though the API does not make this information available to application shaders, it is possible for driver developers to make their compiler aware of it and add some crazy code that can get rid of primitives when it knows that those will never be actually visible. The technique is known as “shader culling”, or “NGG culling”.

This can improve performance in games that have a lot of triangles, because we only calculate the output positions of every vertex before we decide which triangle to remove. We then also remove unused vertices.

The benefits are:

• Reduced bottleneck from the fixed-function HW that traditionally does culling.
• Improved bandwidth use, because we can avoid loading some inputs for vertices we delete.
• Improved shader HW utilization because we can avoid computing additional vertex attributes for deleted vertices.
• More efficient PC (parameter cache) use as we don’t need to reserve output space for deleted vertices and primitives.

If there is interest, I may write a blog post about the implementation details later.

Caveats of shader culling

Due to how all of this reduces certain bottlenecks, its effectiveness very highly depends on whether you actually had a bottleneck in the first place. How many primitives it can remove of course depends on the application. Therefore the exact percentage of performance gain (or loss) also depends on the application.

If an application didn’t have any of the aforementioned bottlenecks or already mitigates them in its own way, then all of this new code may just add unnecessary overhead and actually slightly reduce performance rather than improve it.

Other than that, there is some concern that a shader-based implementation may be less accurate than the fixed-function HW.

• It may leave some triangles which should have been removed. This is not really an issue as these will be removed by fixed-func HW anyway.
• The bigger problem is that it may delete triangles which should have been kept. This can manifest itself by missing objects from a scene or flickering, etc.

Our results with shader culling on RDNA2

Shader culling seems to be most efficient in benchmarks and apps that output a lot of triangles without having any in-app solution for dealing with the bottlenecks. It is also very effective on games that suffer from overtessellation (ie. create a lot of very small triangles which are not visible).

• An extreme example is the instancing demo by Sascha Willems which gets a massive boost
• Basemark sees 10%+ performance improvement
• Doom Eternal gets a 3%-5% boost (depending on the GPU and scene)
• The Witcher 3 also benefits (likely thanks to its overuse of tessellation)
• In less demanding games, the difference is negligible, around 1%

While shader culling can also work on RDNA1, we don’t enable it by default because we haven’t yet found a game that noticably benefits from it. On RDNA1, it seems that the old and new pipelines have similar performance.

Notes about hardware support

• Vega had something similar, but I haven’t heard of any drivers that ever used this. Based on public info I cound find, it’s not even worth looking into.
• Navi 10 and 12 lack some features such as per-primitive outputs which makes it impossible to implement mesh shaders on these GPUs. We don’t use NGG on Navi 14 (RX 5500 series) because it doesn’t work.
• Navi 21 and newer have the best support. They have all necessary features for mesh shaders. We enabled shader culling by default on these GPUs because they show a measurable benefit.
• Van Gogh (the GPU in the Steam Deck) has the same feature set as Navi 2x. It also shows benefits from shader culling, but to a smaller extent.

Closing thoughts

The main takeaway from this post is that NGG is not a performance silver bullet that magically makes all your games faster. Instead, it is an enabler of new features. It lets the driver implement new techniques such as shader culling and new programming models like mesh shaders.

Refresher about the task shader API

The task shader (aka. amplification shader in D3D12) is a new stage that runs in workgroups similar to compute shaders. Each task shader workgroup has two jobs: determine how many mesh shader workgroups it should launch (dispatch size), and optionally create a “payload” (up to 16K data of your choice) which is passed to mesh shaders.

Additionally, the API allows task shaders to perform atomic operations on the payload variables.

Typical use of task shaders can be: cluster culling, LOD selection, geometry amplification.

Expectations on task shaders

Before we get into any HW specific details, there are a few things we should unpack first. Based on the API programming mode, let’s think about some expectations on a good driver implementation.

Storing the output task payload. There must exist some kind of buffer where the task payload is stored, and the size of this buffer will obviously be a limiting factor on how many task shader workgroups can run in parallel. Therefore, the implementation must ensure that only as many task workgroups run as there is space in this buffer. Preferably this would be a ring buffer whose entries get reused between different task shader workgroups.

Analogy with the tessellator. The above requirements are pretty similar to what tessellation can already do. So a natural conclusion may be that we may be able to implement task shaders by abusing the tessellator. However, this introduces a potential bottleneck on fixed-function hardware which we would prefer not to do.

Analogy with a compute pre-pass. Another similar thing that comes to mind is a compute pre-pass. Many games already do something like this: some pre-processing in a compute dispatch that is executed before a draw call. Of course, the application has to insert a barrier between the dispatch and the draw, which means the draw can’t start before every invocation in the dispatch is finished. In reality, not every graphics shader invocation depends on the results of all compute invocations, but there is no way to express a more fine-grained dependency. For task shaders, it is preferable to avoid this barrier and allow task and mesh shader invocations to overlap.

Task shaders on AMD HW

What I discuss here is based on information that is already publicly available in open source drivers. If you are already familiar with how AMD’s own PAL-based drivers work, you won’t find any surprises here.

First things fist. Under the hood, task shaders are compiled to a plain old compute shader. The task payload is located in VRAM. The shader code that stores the mesh dispatch size and payload are compiled to memory writes which store these in VRAM ring buffers. Even though they are compute shaders as far as the AMD HW is concerned, task shaders do not work like a compute pre-pass. Instead, task shaders are dispatched on an async compute queue while at the same time the mesh shader work is executed on the graphics queue in parallel.

The task+mesh dispatch packets are different from a regular compute dispatch. The compute and graphics queue firmwares work together in parallel:

• Compute queue launches up to as many task workgroups as it has space available in the ring buffer.
• Graphics queue waits until a task workgroup is finished and can launch mesh shader workgroups immediately. Execution of mesh dispatches from a finished task workgroup can therefore overlap with other task workgroups.
• When a mesh dispatch from the a task workgroup is finished, its slot in the ring buffer can be reused and a new task workgroup can be launched.
• When the ring buffer is full, the compute queue waits until a mesh dispatch is finished, before launching the next task workgroup.

You can find out the exact concrete details in the PAL source code, or RADV merge requests.

Side note, getting some implementation details wrong can easily cause a deadlock on the GPU. It is great fun to debug these.

The relevant details here are that most of the hard work is implemented in the firmware (good news, because that means I don’t have to implement it), and that task shaders are executed on an async compute queue and that the driver now has to submit compute and graphics work in parallel.

Keep in mind that the API hides this detail and pretends that the mesh shading pipeline is just another graphics pipeline that the application can submit to a graphics queue. So, once again we have a mismatch between the API programming model and what the HW actually does.

Squeezing a hidden compute pipeline in your graphics

In order to use this beautiful scheme provided by the firmware, the driver needs to do two things:

• Create a compute pipeline from the task shader.
• Submit the task shader work on the asyc compute queue while at the same time also submit the mesh and pixel shader work on the graphics queue.

We already had good support for compute pipelines in RADV (as much as the API needs), but internally in the driver we’ve never had this kind of close cooperation between graphics and compute.

When you use a draw call in a command buffer with a pipeline that has a task shader, RADV must create a hidden, internal compute command buffer. This internal compute command buffer contains the task shader dispatch packet, while the graphics command buffer contains the packet that dispatches the mesh shaders. We must also ensure correct synchronization between these two command buffers according to application barriers ― because of the API mismatch it must work as if the internal compute cmdbuf was part of the graphics cmdbuf. We also need to emit the same descriptors and push constants, etc. When the application submits the graphics queue, this new, internal compute command buffer is then submitted to the async compute queue.

Thus far, this sounds pretty logical and easy.

The actual hard work is to make it possible for the driver to submit work to different queues at the same time. RADV’s queue code was written assuming that there is a 1:1 mapping between radv_queue objects and HW queues. To make task shaders work we must now break this assumption.

So, of course I had to do some crazy refactor to enable this. At the time of writing the AMDGPU Linux kernel driver doesn’t support “gang submit” yet, so I use scheduled dependencies instead. This has the drawback of submitting to the two queues sequentially rather than doing everything in the same submit.

Conclusion, perf considerations

Let’s turn the above wall of text into some performance considerations that you can actually use when you write your next mesh shading application.

1. Because task shaders are executed on a different HW queue, there is some overhead. Don’t use task shaders for small draws or other cases when this overhead may be more than what you gain from them.
2. For the same reason, barriers may require the driver to emit some commands that stall the async compute queue. Be mindful of your barriers (eg. top of pipe, etc) and only use these when your task shader actually depends on some previous graphics work.
3. Because task payload is written to VRAM by the task shader, and has to be read from VRAM by the mesh shader, there is some latency. Only use as much payload memory as you need. Try to compact the memory use by packing your data etc.
4. When you have a lot of geometry data, it is beneficial to implement cluster culling in your task shader. After you’ve done this, it may or may not be worth it to implement per-triangle culling in your mesh shader.
5. Don’t try to reimplement the classic vertex processing pipeline or emulate fixed-function HW with task+mesh shaders. Instead, come up with simpler ways that work better for your app.

NVidia also has some perf recommendations here which mostly apply to any other HW, except for the recommended number of vertices and primitives per meshlet because the sweet spot for that can differ between GPU architectures.

Stay tuned

It has been officially confirmed that a Vulkan cross-vendor mesh shading extension is coming soon. Update: it’s here!

While I can’t give you any details about the new extension, I think it won’t be a surprise to anyone that it may have been was the motivation for my work on mesh and task shaders.

Once the new extension goes public, I will post some thoughts about it and a comparison to the vendor-specific NV_mesh_shader extension.

Note, 2022-09-03: updated with a link to the new vendor-netural Vulkan mesh shader extension.

Short intro to NGG

NGG (Next Generation Geometry) is the technology that is responsible for any vertex and geometry processing in RDNA GPUs (with some caveats). Also known as “primitive shader”, the main innovations of NGG are:

• Shaders are aware of not only vertices, but also primitives (this is why they are called primitive shader).
• The output topology is entirely up to the shader, meaning that it can create output vertices and primitives with an arbitrary topology regarless of its input.
• On RDNA2 and newer, per-primitive output attributes are also supported.

This flexibility allows the driver to implement every vertex/geometry processing stage using NGG. Vertex, tess eval and geometry shaders can all be compiled to NGG “primitive shaders”. The only major limiting factor is that each thread (SIMD lane) can only output up to 1 vertex and 1 primitive (with caveats).

The driver is also capable of extending the application shaders with sweet stuff such as per-triangle culling, but this is not the main focus of this blog post. I also won’t cover the caveats here, but I may write more about NGG in the future.

Mapping the mesh shader API to NGG

The draw commands as executed on the GPU only understand a number of input vertices but the mesh shader API draw calls specify a number of workgroups instead. To make it work, we configure the shader such that the number of input vertices per workgroup is 1, and the output is set to what you passed into the API. This way, the FW can figure out how many workgroups it really needs to launch.

The driver has to accomodate the HW limitation above, so we must ensure that in the compiled shader, each thread only outputs up to 1 vertex and 1 primitive. Reminder: the API programming model allows any shader invocation to write any vertex and/or primitive. So, there is a fundamental mismatch between the programming model and what the HW can do.

This raises a few interesting questions.

How do we allow any thread to write any vertex/primitive? The driver allocates some LDS (shared memory) space, and writes all mesh shader outputs there. At the very end of the shader, each thread reads the attributes of the vertex and primitive that matches the thread ID and outputs that single vertex and primitive. This roundtrip to the LDS can be omitted if an output is only written by the thread with matching thread ID.

What if the MS workgroup size is less than the max number of output vertices or primitives? Each HW thread can create up to 1 vertex and 1 primitive. The driver has to set the real workgroup size accordingly:
hw workgroup size = max(api workgroup size, max vertex count, max primitive count)
The result is that the HW will get a workgroup that has some threads (invocations) that execute the code you wrote (the “API shader”), and then some that won’t do anything but wait until the very end to output their up to 1 vertex and 1 primitive. It can result in poor occupancy (low HW utilization = bad performance).

What if the shader also has barriers in it? This is now turning into a headache. The driver has to ensure that the threads that “do nothing” also execute an equal amount of barriers as those that run your API shader. If the HW workgroup has the same number of waves as the API shader, this is trivial. Otherwise, we have to emit some extra code that keeps the extra waves running in a loop executing barriers. This is the worst.

What if the API shader also uses shared memory, or not all outputs fit the LDS? The D3D12 spec requires the driver to have at least 28K shared memory (LDS) available to the shader. However, graphics shaders can only access up to 32K LDS. How do we make this work, considering the above fact that the driver has to write mesh shader outputs to LDS? This is getting really ugly now, but in that case, the driver is forced to write MS outputs to VRAM instead of LDS.

How do you deal with the compute-like stuff, eg. workgroup ID, subgroup ID, etc.? Fortunately, most of these were already available to the shader, just not exposed in the traditional VS, TES, GS programming model. The only pain point is the workgroup ID which needs trickery. I already mentioned above that the HW is tricked into thinking that each MS workgroup has 1 input vertex. So we can just use the same register that contains the vertex ID for getting the workgroup ID.

Conclusion, performance considerations

The above implementation details can be turned into performance recommendations.

Specify a MS workgroup size that matches the maximum amount of vertices and primitives. Also, distribute the work among the full workgroup rather than leaving some threads doing nothing. If you do this, you ensure that the hardware is optimally utilized. This is the most important recommendation here today.

Try to only write to the mesh output array indices from the corresponding thread. If you do this, you hit an optimal code path in the driver, so it won’t have to write those outputs to LDS and read them back at the end.

Use shared memory, but not excessively. Implementing any nice algorithm in your mesh shader will likely need you to share data between threads. Don’t be afraid to use shared memory, but prefer to use subgroup functionality instead when possible.

What if you don’t want do any of the above?

That is perfectly fine. Don’t use mesh shaders then.

The main takeaway about mesh shading is that it’s a very low level tool. The driver can implement the full programming model, but it can’t hold your hands as well as it could for traditional vertex processing. You may have to implement things (eg. vertex inputs, culling, etc.) that previously the driver would do for you. Essentially, if you write a mesh shader you are trying to beat the driver at its own game.

Wait, aren’t we forgetting something?

I think this post is already dense enough with technical detail. Brace yourself for the next post, where I’m going to blow your mind even more and talk about how task shaders are implemented.

Problems with the old geometry pipeline

The problem with the traditional vertex processing pipeline is that it is mainly designed assuming several fixed-function hardware units in the GPU and offers very little flexibility for the user to customize it. The main issues with the traditional pipeline are:

• Vertex buffers and vertex shader inputs are annoying (especially from the driver’s perspective) and input assembly may be a bottleneck on some HW in some cases.
• The user has no control over how the input vertices and primitives are arranged, so the vertex shader may uselessly run for primitives that are invisible (eg. occluded, or backfacing etc.) meaning that compute resources are wasted on things that don’t actually produce any pixels.
• Geometry amplification depends on fixed function tessellation HW and offers poor customizability. (Or GS, which is even worse.)
• The programming model allows very poor control over input and output primitives. Geometry shaders have a horrible programming model that results in low HW occupancy and limited topologies.

The mesh shading pipeline is a graphics pipeline that addresses these issues by completely replacing the entire traditional vertex processing pipeline with two new stages: the task and mesh shader.

What is a mesh shader?

The mesh shader is a compute-like stage which allows the application to fully customize its inputs and outputs, including output primitives and their topology.

• Mesh shader vs. Vertex shader: A mesh shader is responsible for creating its output vertices and primitives. In comparison, a vertex shader is only capable of loading a fixed amount of vertices, doing some processing on them and has no awareness of primitives.
• Mesh shader vs. Geometry shader: As opposed to a geometry shader which can only use a fixed “strip” output topology, a mesh shader is free to define whatever topology it wants. You can think about it as if the mesh shader produces a small indexed triangle list.

What does it mean that the mesh shader is compute-like?

You can use all the sweet good stuff that compute shaders already can do, but vertex shaders couldn’t, for example: use shared memory, run in workgroups, rely on workgroup ID, subgroup ID, etc.

The API allows any mesh shader invocation to write to any vertex or primitive. The invocations in each mesh shader workgroup are meant to co-operatively produce a small set of output vertices and primitives (this is sometimes called a “meshlet”). All workgroups together create the full output geometry (the “mesh”).

What does the mesh shader do?

First, it needs to figure out how many vertices and primitives it wants to create and pass those numbers to SetMeshOutputsEXT (SetMeshOutputCounts in D3D12), then it can write to its output arrays. How it does this, is entirely up to the application developer. Though, there are some performance recommendations which you should follow to make it go fast. I’m going to talk about these more in the next post.

The input assembler step is entirely eliminated, which means that the application is now in full control of how (or if at all) the input vertex data is fetched, meaning that you can save bandwidth on things that don’t need to be loaded, etc. There is no “input” in the traditional sense, but you can rely on things like push constants, UBO, SSBO etc.

For example, a mesh shader could perform per-triangle culling in such a manner that it wouldn’t need to load data for primitives that are culled, therefore saving bandwidth.

What is a task shader aka. amplification shader?

The task shader is an optional stage which percedes the mesh shader (and is also compute-like). Each task shader workgroup has two main purposes:

• Decide how many mesh shader workgroups need to be launched.
• Create an optional “task payload” which can be passed to mesh shaders.

“Geometry amplification” is achieved by choosing to launch more (or less) mesh shader workgroups. As opposed to the fixed-function tessellator in the traditional pipeline, it is now entirely up to the application how to create the vertices.

While you could re-implement the old fixed-function tessellation with mesh shading, this may not actually be necessary and your application may work better with a simpler custom algorithm.

Another interesting use case for task shaders is per-meshlet culling, meaning that a task shader is a good place to decide which meshlets you actually want to render and eliminate entire mesh shader workgroups which would otherwise operate on invisible primitives.

• Task shader vs. Tessellation. Tessellation relies on fixed-function hardware and makes users do complicated shader I/O gymnastics with tess control shaders. The task shader is straightforward and only as complicated as you want it to be.
• Task shader vs. Geometry shader. Geometry shaders operate on input primitives directly and replace them with “strip” primitives. Task shaders don’t have to be directy get involved with the geometry output, but rather just let you specify how many mesh shader workgroups to launch and let the mesh shader deal with the nitty-gritty details.

Usefulness

For now, I’ll just discuss a few basic use cases.

Meshlets. If your application loads input vertex data from somewhere, it is recommended that you subdivide that data (the “mesh”) into smaller chunks called “meshlets” (it is recommended that you do this at asset creation time). Then you can write your shaders such that each mesh shader workgroup processes a single meshlet.

Procedural geometry. Your application can generate all its vertices and primitives based on a mathematical formula that is implemented in the shader. In this case, you don’t need to load any inputs, just implement your formula as if you were writing a compute shader, then store the results into the mesh shader output arrays.

Replacing compute pre-passes. Many modern games use a compute pre-pass. They launch some compute shaders that do some pre-processing on the geometry before the graphics work. These are no longer necessary. The compute work can be made part of either the task or mesh shader, which removes the overhead of the additional dispatch and barrier.

Note that mesh shader workgroups may be launched as soon as the corresponding task shader workgroup is finished, so mesh shader execution (of the already finished tasks) may overlap with task shader execution, removing the need for extra synchronization on the application side.

Conclusion

Thus far, I’ve sold you on how awesome and flexible mesh shading is, so it’s time to ask the million dollar question.

Is mesh shading for you?

The answer, as always, is: It depends.

Yes, mesh and task shaders do give you a lot of opportunities to implement things just the way you like them without the stupid hardware getting in your way, but as with any low-level tools, this also means that you get a lot of possibities for shooting yourself in the foot.

The traditional vertex processing pipeline has been around for so long that on most hardware it’s extremely well optimized because the drivers do a lot of optimization work for you. Therefore, just because an app uses mesh shaders, doesn’t automatically mean that it’s going to be faster or better in any way. It’s only worth it if you are willing to do it well.

That being said, perhaps the easiest way to start experimenting with mesh shaders is to rewrite parts of your application that used to use geometry shaders. Geometry shaders are so horribly inefficient that it’ll be difficult to write a worse mesh shader.

How is mesh shading implemented under the hood?

Stay tuned for the next posts if you are curious about that! I’m going to talk about how mesh and task shaders are implemented in a driver. This will shed some light on how these shaders work internally and why certain things perform really badly.

References

• Introduction to Turing Mesh Shaders
• Using Mesh Shaders for Professional Graphics
• Advanced API Performance: Mesh Shaders
• DirectX 12 mesh shader API
• Khronos blog: Mesh shading for Vulkan

Note, 2022-09-03: updated with a link to the new vendor-netural Vulkan mesh shader extension.