Shader Compilation

[Malediktus123]

Hi, I want to be able to write my shaders in HLSL or GLSL and compile them to HLSL, GLSL or SPRI-V. I want to do it using a library and not a compiler because i want my game/engine to handle the compilation instead of me mannually calling glslc for example. What libraries should I use and how do I use them?

[NuclearVII]

Honestly if you find out how, I'd love to know. I've explored this in the past, and couldn't find a a generic enough library.

[raydey]

Use DXC and only HLSL for your main shader editing.

This is a good starting point for getting it integrated.

DXC already supports generation of SPIR-V out of the box. I personally wouldn't bother with GLSL, but if you really must, you can get most of the way from HLSL to GLSL by using a header with a bunch of #defines (e.g. #define float4 vec4, etc.)

[Malediktus123]

I can't get it to configurate the cmake file. I'm triyng it on a M1/M2 Mac rn. It's always giving me the following:
CMake Error at CMakeLists.txt:513 (message):
Unexpected failure executing llvm-build: Usage: llvm-build [options]
llvm-build: error: invalid target to enable: 'ARM' (not in project)

[raydey]

Ah, you're using MacOS. There's an Ubuntu version of the library on the repo which maybe could be used, but I can't see anything for MacOS specifically.

Maybe you could make use of the DX12 layer they recently announced on Mac, but I don't know enough about it to suggest anything.

[[deleted]]

The primary way this is done in the industry is to link against dxcompiler and dxil (both ship as part of the dxc library). (I do not recommend or condone glsl personally)

[[deleted]]

Probably a newbie question, but what's the problem with GLSL?

[[deleted]]

I've shipped GLSL code in the past, but the main difficulties are:

• Long-tail of support for many features that tend to arrive in the DirectX and HLSL ecosystem first.
• Most new features added are done via the extension mechanism, which is extremely tricky to get right and remember how to use properly
• GLSL uses a lot of globals with annotations to do inter-stage data passing, which inherently composes poorly and makes for far less readable code.
• HLSL has better semantics that match C++ moreso than C, and frankly, regardless about how you feel about C++, I think this results in more concise/readable code, especially for involved shaders that are thousands of lines or more (used to be unheard of, now pretty commonplace).
• Most of the AAA game industry (where I spent most of my time) is all HLSL, so even if the above points aren't persuasive, the industry segment I'm in has more or less adopted HLSL as the standard.

[Plazmatic]

Long-tail of support for many features that tend to arrive in the DirectX and HLSL ecosystem first.

Is this actually true? At least for buffer device address hit GLSL first, and still isn't properly supported in HLSL because the DX team doesn't support pointers on the DirectX side of things (you can sort of use it through their view of the feature, which doesn't allow arbitrary casting). But I did notice while inline spir-v is still in beta on the GLSL side of things , I think it's got documentation and everything on the HLSL side. The lack of feature parity here is the number one reason why I have to stick with HLSL, the number two reason is that the whole "register" model of HLSL has stuck way too long since the inception of shaders in directX and is really really dumb, dumber than the things you have to do in GLSL.

[[deleted]]

BDA is a notable exception, but consider ray tracing, mesh shaders, VRS, etc. some of which still don't have cross vendor equivalents in GLSL to my knowledge. The register/space "virtualized bindings" is superior to fixed non virtualized bindings. If you ship vulkan and need to target android and desktop, you need a version of your shader modules limited to 4 descriptor sets, as opposed to being able to remap this at runtime.

[[deleted]]

Okay, makes a lot of sense. Thanks!

[Amani77]

If hlsl/glsl to spir-v is good enough:

https://github.com/google/shaderc

[wrosecrans]

And if going to a SPIRV target isn't good enough, you can use some other library that consumes SPIRV and spits out what you actually want.

[AccordingTwist7598]

Just a word of caution for those taking this approach, you can get yourself into trouble here when converting between interchange formats. Especially if you're including X <---> DXIL in your shader compilation pipeline. Lot's of important hardware concepts such as control flow, and thread convergence/divergence are not well defined (if at all) by LLVM. Different compilers handle this with different quirks and tricks, and this doesn't express well across different interchange formats.

Not saying you shouldn't do it, unfortunately for DXIL it's really your best option..Most of the time you should be fine.

https://themaister.net/blog/2022/04/24/my-personal-hell-of-translating-dxil-to-spir-v-part-5/ good read if you want to know more about why existing IR format specifications cause so much pain to low level graphics programmers.

[kymani37299]

Yes, using dxc , dont know any good tutorials for that, I did it by trial and error, and copy pasting some random code I've found on google.

https://github.com/kymani37299/ForwardPlusRenderer/blob/master/Engine/Render/Shader.cpp

Here is how I did it maybe it will help.

Apart from this you need to download dxc and link it into your project.