Wish list: compile-time SPIRV-Cross

[parasyte]

I came across the discussion in gfx-rs/wgpu-rs#44 and it lead me down a bit of a rabbit hole. TL;DR: Make the runtime shader compiler spirv_cross optional in the backends that need it.

Longer version is as follows.

I like vk-shader-macros because it moves the SPIR-V compiler to compile-time. But nearly all of the gfx backends depend on spirv_cross (the exceptions being vulkan and empty 😉). You end up with a shader compiler statically built into your application in most cases, anyway.

There are some valid uses for using a shader compiler at runtime. And then there are some valid uses for compiling shaders entirely at compile time. My pedantic nature lead me down the path of trying to optimize a toy like "hello-triangle", only to find that at least 7% (about 220 KB) of the binary size is spirv_cross; a large chunk of code which arguably should not exist in this kind of trivial executable.

The proposal here is to add a macro that will compile SPIR-V to the native shader language at compile-time, to complement Device::create_shader_module(). The macro should still accept a &[u32] of SPIR-V, and developers can choose their preferred method of creating a shader module.

[parasyte]

See also rust-gamedev/wg#23

Seems like this wish list item might be a better fit for javelin.

[grovesNL]

It would be great if we could do this, but I don't think it's going to be (easily?) possible for most cases unfortunately. Usually we need information which is only provided later at run-time in order to override certain values in the SPIR-V before converting the provided SPIR-V to MSL/GLSL/HLSL. The exception to this is the Vulkan backend which receives SPIR-V directly.

For example, we need to be able to specialize SPIR-V shaders based on specialization constants provided at run-time. But not all specialization constants can be handled natively in MSL with MTLFunctionConstant, so we actually need to regenerate MSL from SPIR-V with the constant provided at run-time. Otherwise we'd need to know all possible specialization constant values at compile-time so we can generate each shader.

The specialization constant example is one case, but there are a lot more cases: overriding resource bindings, etc. Hopefully Javelin will bring the binary size down further in general, but I'm not sure that we can rely on doing most of this at compile-time only.

[kvark]

Another example is that shader translation depends on the pipeline layout, which you build at run-time. We could think of a mode of operation where the spirv_cross is not included in the build, and the non-Vulkan backends require your shader/pipeline to already be in the provided pipeline cache... but that seems super tricky. I think just getting Javelin rolling (assuming it's a much more lightweight dependency than spirv_cross) would be preferable.

[swiftcoder]

Feels like it's pretty rare (at least for smaller stuff) that I don't also know things like my pipeline structure up front. I wonder if we could let one specify all of this structure at compile-time (i.e. via proc macro) for straightforward cases.

[kvark]

So far we got:

• pipeline layout
• specialization constants
• some of the pipeline state, like the rasterization mode

@swiftcoder overall, it's hard to make this subset distinctive from the stuff we provide at pipeline creation. Which leads me back to the idea of using the pipeline cache and making spirv_cross dependency optional in the backends by enabled by default. This way, you could theoretically prepare the pipeline caches on the development machine, save them, and then ship your product with spirv_cross feature disabled.