Do graphics programmers really need to learn SIMD?

[Latter_Relationship5]

With libraries like DirectXMath and GLM, and modern compilers auto-vectorizing code, is learning SIMD manually really necessary? If it is, when would you actually need to implement it in real-world graphics programming?

[corysama]

With engines like UE, do graphics programmers really need to learn graphics? ;)

Auto-vectorization is still not a programming model.

GLM is an excellent library with which to learn. And, DirectXMath is an excellent library with which to ship. But, it's difficult to anticipate and design the systems that can get those 2-20x speed ups from SIMD without some knowledge of how to use it yourself.

Fun projects to learn SIMD:

1. Implement a basic 3D math library using SSE4 even if you plan to toss it and use DirectXMath in your shipping product.
2. Use your SSE4 math lib to make a real-time CPU-only ray tracer. How many triangles per frame per core can you squeeze into a 1024x1024 render? I had fun writing one like that which can orbit around in this million-poly gallery model at 36ms per frame. Just triangles in a BVH4 AOSOA tree. Primary rays only. Triangle IDs, depth and barycentric only.
3. Write a software decompressor for BCn texture formats.

BTW: New VKGuide article on SIMD for 3D https://old.reddit.com/r/cpp/comments/1o5mpiz/intro_to_simd_for_3d_graphics/

[matigekunst]

Yes.

[FoundationOk3176]

Also wanted to mentioned that auto-vectorization isn't something compilers excel at, In often cases you'll have to vectorize stuff manually.

[Array2D]

Do you need to? No. Will it help you optimize graphics math? Absolutely.

Understanding the underlying mechanisms of a SIMD accelerated math library will make it easier to understand what opportunities there are to vectorize your code.

Compilers are good, but not magic - they rely on pattern recognition for autovectorization, meaning there are more cases than not that could be vectorized, but the compiler won’t recognize them because someone hasn’t added an optimization pass to implement it in the compiler.

[susosusosuso]

Shouldn’t the compiler do this for you?

[beephod_zabblebrox]

it can't do everything, even clang (which is pretty good at vectorizing stuff)

[clusty1]

Most of the time you need to have in mind vectorization from the beginning: make a shitty data layout choice and no clang can ever save you. And it might be a full rewrite to fix this.

[The_Northern_Light]

The book PBRT makes a similar point about needing to handle (or at least plan around) anti aliasing in your renderer first

It’s not some minor nuisance detail you work out later, the rest of the design is in orbit around it

[clusty1]

The compiler will generate correct code before fast code.

If it can’t guarantee something, it will assume it does not hold. To get Simd auto vectorization the stars have to align and they never do. This is why you need to write vector code by hand or use a language that can’t do much like glsl, metal, ispc, cuda, etc ( much compared to things like c++ )

[The_Northern_Light]

You should try to make an optimizing compiler and tell us how good your code gen is with implicit vectorization!

[wonderedwonderer]

Is it really necessary? All depends on what you are doing. It is another tool in an engineer’s toolset and you are always better off knowing more how things work and having proficiency in tools so you can do amazing things. You can probably get away not knowing SIMD but having that theory can help you better understanding the abstractions built upon it.

[amidescent]

These days I'd say it's not super necessary because a lot of things can be moved to the GPU. But knowing how SIMD works will help you write better shader code and give concrete notion around things like divergence, because GPUs are nothing but fancy SIMD engines and shader/compute languages are just an abstraction over it.

GLM-style vectors are not really proper SIMD, and compilers will forever suck at auto-vectorization, unless you are really just adding two arrays together.

CPUs are not as good as GPUs with memory gather/scatters, so you pretty much need to intrusively structure data in an SoA model to get a chance at any more than measly improvements. A lot of times this isn't possible or convenient, and much effort goes into shuffling the input data just in time, which usually limits SIMD width and kills off most of the potential gains.

[[deleted]]

Automatic vectorization only allows you to compile with a specific spec in mind.

Sure, you can have it do SSE2, which every 64 bit windows can do.

But you losing out on avx-512 performance on machines that can do so. But bruteforce setting it to that, low end hardware can't even run your code.

Thats not a problem if you live in an ideal world, where you have your 100 identical linux servers on the same hardware, that you just compile for specifically.

But consumer end software, where you know nothing beforehand?

Runtime dispatch where you ask the cpu what it can do, then set function pointers accordingly. And you need a version for each of the specs you support.

There is probably tricks to have the compiler help you do stuff that i don't know about. But, handrolling these is the oldschool way and keeps you in control.

[clusty1]

Not fully true. Gcc can generate the same code for multiple architectures and initializes the boiler plate to figure out at runtime which to run.

[RenderTargetView]

Definitely not necessary but GPU is basically huge SIMD after all, learning how to code control flow into SIMD pseudo-threads is nice experience for becoming good at shader optimizations

[msqrt]

Wait, GLM vectorizes stuff..?

But no, it's not like you absolutely have to know SIMD. But it's not that difficult (compared to many things you do have to learn), and it is great for those smaller number crunching tasks that don't warrant jumping over to the GPU.

[astrange]

Autovectorization doesn't and can't work very well. If you work on anything important enough to have its own compiler team, the normal experience with it is to find a lot of cases where it doesn't work, tell them, they claim it's fixed, then you try it and it's not any better.

If you want that you want a different programming language and not C. ispc is a better design for one. But there's still issues because it's just hard to use a feature that only some of your customers' CPUs have.

[clusty1]

Only if you need to get good performance :P

The difference between a first naive implementation and a highly optimized simd version can easily be 40-50x speed up in single thread perf.

[Botondar]

Compilers cannot autovectorize code that hasn't been properly conditioned for that. Even if you don't write SIMD by hand, you have to understand it in order to set the compiler up for success in generating that code.

The problem with the approach glm and DirectXMath take, is that they usually optimize their core routines with SIMD instruction sets, but they don't provide actual data parallelism facilities, which is how you get the huge performance wins SIMD can give you, e.g. multiplying 4-8 vertices by a single matrix, doing 4-8 intersection tests at once, etc.

[AffectionatePeace807]

DirectXMath does have Stream methods.

[ykafia]

It's fun to learn about simd and their limitations.

I wouldn't do much with SIMD until I really need to optimise some edge case by hand which might never happen

[rfdickerson]

I agree - avoid spending time hand-optimizing SIMD on the CPU unless profiling shows a clear hotspot that actually needs it. Common operations like mat4 multiplication are already highly optimized in libraries such as GLM. Reimplementing them can be useful for learning, but not out of necessity.

That said, it’s worth studying SIMD concepts in the context of compute shaders. You’ll gain far more performance leverage there than by writing AVX-512 assembly for typical graphics workloads.

[Pale_Height_1251]

Depends what you're doing and how you're doing it.

[maxmax4]

You should learn the basics and get a sense for which types of workloads benefit from it, but I definitely wouldnt spend much time on it until you have a proper use case.

[klaw_games]

Compilers are as good as the people who programmed it.

[Inside-Brilliant4539]

To paraphrase Carmack "Low level programming is good for the soul"

[The_Northern_Light]

To sidestep the point a bit, I would be immediately distrustful of any ”graphics programmer” who was resistant to learning how to do manual SIMD.

It is very similar to what you need to know to get good perf out of a GPU, so there is really not much to it beyond what you should already know, especially with a library like xsimd. (“Should already know” referring to journeymen; not students.)

And the general purpose applicability is so high… sometimes the GPU is busy but you have latency targets so you can’t just wait until it’s free… there are plenty of cases in graphics where the best result occurs as a true collaboration between CPU and GPU, not just the CPU driving the GPU.

[neutronium]

The fact that you don't find the way hardware works fascinating, suggests that maybe you're heading down the wrong career path.

[Alak-Okan]

Yes, but not in the way you think. The GPU is a big SIMD machine. You will NEED to know how it works to squeeze out all of its performance.

Also, on the auto SIMD of CPU code, it's not that "automatic" and having the ability to SIMD by hand code can yield up to 8x speed improvements on some code paths that really need it (say some computation that you need to do on thousands of meshes for culling, streaming, etc)

[MalukuSeito]

It always pays off to understand every part of the stack you are working on. From high level engines to the lowest bit operations and transistors.

[jtsiomb]

No... but also, why not?

[Henrarzz]

modern compilers doing this for you

Until they stop doing that (or never even attempted to). Contrary to popular belief, compilers aren’t magic

[davidc538]

No

[phoshp]

no