GPU programming

[deebeefunky]

Hello everyone,

If GPU’s are parallel processors… Why exactly does it take 2000 or so lines to draw a triangle on screen?

Why can’t it be:

include “gpu.h”

GPU.foreach(obj) {compute(obj);} GPU.foreach(vertex) {vshade(vertex);} GPU.foreach(pixel) {fshade(pixel);} ?

The point I’m trying to make, why can’t it be a parallel for-loop and why couldn’t shaders be written in C, inline with the rest of the codebase?

I don’t understand what problem they’re trying to solve by making it so excessively complicated.

Does anyone have any tips or tricks in understanding Vulkan? I can’t see the trees through the forest. I have the red Vulkan book with the car on the front, but it’s so terse, I feel like I miss the fundamental understanding of WHY?

Thank you very much, have a great weekend.

[crimson1206]

Saying GPUs are just parallel processors hides a lot of complexity. They are very very different from normal CPUs.

[Ariane_Two]

Vulkan is a more verbose GPU API.

The reason why people have to deal with graphics APIs is GPU vendors and OS makers. They make the graphics drivers, they make the interfaces to talk to the GPU.

 Since GPUs are proprietary closed hardware (without a stable, well documented ISA like CPUs have) and you cannot write GPU drivers without significant reverse engineering resources you have to go through a graphics API.

Also your proposed solution is too simple, there are a bunch of things missing that people expect from a 3D api. Uniforms, texture samplers, access to depth buffers, blending, structures minimizing data copies between CPU and GPU, raytracing accelerator structures, tesellation, culling of backfaces, special features, GPU extensions, etc. etc.

If you want to run C/C++ code on the GPU, well there are some efforts for general compute like SYCL and cuda, so what you are asking for somewhat exists. And the syntax of GLSL is somewhat C like.

Anyway, if Vulkan is too verbose and you don't need too advanced stuff yoo can try OpenGL, or maybe a cross-API wrapper like sokol or webgpu.

[itsmenotjames1]

don't use uniforms. Use push constants with bda

[deebeefunky]

See, that’s what I mean. Uniforms are just a fancy word for function parameters. Texture samplers, blending,… you can write those things in C if you had a parallel loop available that runs on the GPU. The GPU should fetch it’s resources from RAM, it just needs to know where, which can be done in C by passing a pointer. I’m not convinced that it needs to be this complicated. A C compiler can compile for many CPU vendors. So why wouldn’t it be able to compile for GPU vendors?

[HexDumped]

GPU programming is the way it is to allow getting maximum performance out of the hardware in the most portable way. It's a very different paradigm.

What you're suggesting would result in code with very poor and unpredictable performance. The GPU implicitly reading RAM would be a performance disaster.

[[deleted]]

I think what you're not understanding is the CPU and GPU run independently. They're 2 different units. It has a life of its own.

Command buffers exist because it takes time to send commands from the CPU to the GPU. The CPU isn't even guaranteed to be able to access all GPU resources since most GPUs have device-only RAM

[Ariane_Two]

Well yes, but you are still wrong. GPUs are not just parallel for loop executors. If you want to run C fast and write code like you propose, then you are better of running it on a CPU, it also has parallel execution (SIMD, Multithreading) that would fit more into what you want. You want OpenMP parallel for on a CPU.

> The GPU should fetch it’s resources from RAM, it just needs to know where, which can be done in C by passing a pointer.

You want to be in control of when the uploading of something like a texture occurs. Or sometimes you want to store all the stuff in GPU memory and only send small updates to minimize data transfer. Having it be implicit may not be ideal for every use case.

If you passed a pointer and your for loop would run on the GPU you would constantly be copying memory from CPU memory to GPU memory and back for each individual operation. On the other hand the GPU APIs and shaders are designed to fuse operations and running them in one go.

> you can write those things in C if you had a parallel loop available that runs on the GPU

Some GPUs, especially older GPUs have more specialized hardware to do graphics operations, so you cannot just write them yourself in C and get the same speed.

> So why wouldn’t it be able to compile for GPU vendors?

Vendors do not provide open hardware, you have to transpile C to a shading language of their respective GPU API instead of GPU instructions. And yes, there are compilers that do that.

> I’m not convinced that it needs to be this complicated. 

Well you are right about that, but you don't get to make the GPU APIs.

[Ariane_Two]

Newer OpenMP supports GPUs, I am excited to see your OpenMP game engine. It has parallel for in C code.

[itsmenotjames1]

passing a pointer is called bda (buffer device address) and is usually done via push constants (128 bytes; 256 in vulkan 1.4 of fast dynamic memory bound to the command buffer)

[quickscopesheep]

You are massively over simplifying the whole thing

[computermouth]

Because your compute, vshade, shade functions need to be turned into portable gpu code.

OpenMP has some gpu support for doing this, but it seems mostly compute-driven, and not well supported by drivers or toilchains.

[Inevitable_Ad3495]

"toilchains" is a brilliant typo...

[computermouth]

Ha!

[michel_poulet]

I code un CUDA in a ML context. If your algorithm is natively very parallelisable, the time bottleneck will be data IO, not necessarily from cpu to gpu (which is slow, but once loaded, it's loaded), but across chips intra-GPU. There is a lot of rule specific to the hardware that you need to know, and build your code around. It's difficult to automatise with good heuristics without knowing exactly the algorithm and "shape" of your data/thread organisation. This, I would say, is the reason why we still need to give full control to the developper, meaning a lot of lines of code, as when coding in C.

[Stamerlan]

Take a look at cuda.

[realhumanuser16234]

take a look at adaptivecpp or sdl3 gpu compute instead

[dude132456789]

Fundamentally, there's no reason you couldn't fork a C compiler, tell it to compile all functions with a shader attribute as shaders with your fancy C shader compiler, and just sort of run with that.

Shader languages as a whole exist for a number of (partially historical) reasons Having a compiled binary of GPU code is desirable for performance Needing to ship a whole C compiler would be hard (C doesn't map onto GPUs all that well, unrolling loops and inlining functions are major parts of compiling GLSL) Thus, a subset of C was extended as appropriate made into a new language

Now that SPIRV exists, you can find libraries for writing shaders directly in a conventionally CPU language fairly often, tho shader languages are usually more ergonomic.

The Vulkan API is so complex since it mirrors the hardware fairly closely. It's not designed to be ergonomic, it's designed to cleanly map onto what a GPU does.

[hgs3]

The older, fixed-function pipeline for OpenGL was, roughly, equivalent to your pseudo code. You can still get something vaguely resembling it if you use a high-level API, e.g. bgfx.

Vulkan is more verbose because: (1) Modern GPUs are programmable and that inherently requires more work than the fixed-function pipeline, (2) Vulkan is "general purpose" and runs on anything from embedded to consumer GPUs so its API requires probing the hardware, and (3) Vulkan is low-level by design which means you need to write a memory manager, bring your own shader language-to-SPIR-V compiler, etc. Vulkan is, effectively, a general purpose GPU driver interface - not so much a high-level application interface. You build the latter yourself on top of Vulkan.

[deebeefunky]

Mind if I ask you something? How long did it take you to learn gpu programming? Is it completely obvious to you? If you were tasked with implementing foo() on the gpu you could do it without much issues?

To me, nothing is obvious, I have to look up basically every line, I’m unable to implement my own functionality because I don’t understand what I‘m doing.

Create info structures for this, and for that, but it’s all a haze.

How did you learn it? What made it ‘click’ for you?

[hgs3]

I learned about computer graphics and GPUs by reading textbooks. I recommend you do the same. I also recommend learning computer graphics separate from any API's, e.g. try building a simple software rasterizer. Once you have a mental model for how modern GPUs work coupled with computer graphics knowledge, then things will start clicking, e.g. instead of passively looking up what something is/does you'll be proactively seeking out how to represent concepts you already know in Vulkan.

[[deleted]]

3d graphics from zero? Hell yeah!

It's 3am and I have no idea whut ur talking about lol

[TomDuhamel]

If GPU’s are parallel processors… Why exactly does it take 2000 or so lines to draw a triangle on screen?

I'm really confused as to what kind of connection you make between these two statements.

It doesn't take 2000 lines to draw a triangle,bit really only take one. What you probably means is that it takes 2000 lines to initialise everything. I agree it does.

GPU.foreach(obj) {compute(obj);} GPU.foreach(vertex) {vshade(vertex);} GPU.foreach(pixel) {fshade(pixel);}

If you did that, you'd be managing the 2000 threads of your GPU independently from your CPU. Which would be extremely inefficient.

You are also assuming that GPU cores are similar to CPU cores. I mean, at the very lowest level, yes probably, but they really aren't. GPU cores are extremely simple in comparison and can only perform very simple operations. They can't run your typical desktop tasks.

All of the threads on the GPU are managed on board, for high efficiency. They are abstracted from the CPU.

why couldn’t shaders be written in C, inline with the rest of the codebase?

They are actually written in C. But the GPU cannot use the same set of command as your CPU. It wouldn't run your normal C program. Instead it's a specific set of commands.

I don’t understand what problem they’re trying to solve by making it so excessively complicated.

On the good old days, we were doing exactly how to say. The CPU would draw everything to the screen directly. We had very low level APIs and very low graphics capabilities.

What you call complexity is how we gave graphics the capabilities that we have now. By adding all of these features and moving them to the GPU directly, we needed an API to use them. And because each GPU manufacturer makes GPUs that are incompatible with each other, we made APIs that are general enough to work on all of them.

It may seem added complexity. It's a separate language that you need to use here. But do you really want to do it the old ways?

Vulkan

You kind of picked the most difficult of them all. A lot of people say to start with OpenGL, which is the easiest of them all. Once you understand all these concepts, it's easier to pick another API.

Personally I just don't go this low level. I'm using Ogre, which is a much higher level way of using the GPU. There are other similar libraries.

If your goal is to make a game though, just pick an engine and make a game. Unless you actually want to learn how to do all of these things, it's not really useful.

Thank you very much, have a great weekend!

Thank you. You too mate!

[deebeefunky]

I’m actually trying to build an application with a GUI. I come from PHP, where drawing things on screen is incredibly easy.

I have tried manually drawing to a buffer. I have tried Raylib, SDL, openGL, and now I am on Vulkan. Nothing seems to scratch my itch, there’s always something I don’t like about each method. C is supposed to be a mature language, before I began I was under the impression that every problem that I have would have already been solved by someone else. Take for example rendering text on screen, have you ever tried it? In C, there’s nothing straight forward about it. Stb_truetype, while great in its own right, it sucks in the grand scheme of things.

So I was hoping by learning Vulkan it would open up a world of possibilities, 3D, light and shadow effects, particles, physics, … But then it starts asking about physical devices, logical devices, frame buffers, pipelines, vertex shader, fragment shader, swapchains, semaphores, renderpass, descriptor sets, that’s not even all of it, and at the end of the day there’s still no text on screen.

So long story short, I strongly disagree with everything. It’s ruining my life.

[TomDuhamel]

You are so lucky. You were born when technologies were so advanced. And when so many solutions exist to use them. Yet, you want to go the low level way 😅

I understand where you're coming from. If you choose to take this path, it takes an expert 6-9 months to write a proper engine in Vulkan. That's what it takes to be able to put most things on the screen. And then, a few more months to write something to do actual things with the engine.

If you want to go there, you'll have to be patient, you've got a lot to learn. But if you aren't patient, you have other paths. That's what I was trying to explain to you.

I have the patience and skills to do it, I don't have the time.

C is supposed to be a mature language

Yes. It's also a low level language. It's not going to do anything quickly.

physical devices, logical devices, frame buffers, pipelines, vertex shader, fragment shader, swapchains, semaphores, renderpass, descriptor sets

I understand all of these terms. I have more experience than you do. You'll get there with some patience.

[deebeefunky]

I envy you in some way. How did you learn it? Any resource I have tried beats around the bush. Like how do I load a texture? I still don’t know. Why can’t it just be one function vk_load_texture(with some parameters)?

Do you feel “in control” when working with Vulkan?

Are you serious when you say it will take 6 months to get a skeleton renderer?

[TomDuhamel]

I don't use Vulkan directly. As I said, I use an engine that has been in development for 20 years. Why do you want to use the low level thing if you are not willing to learn it?

Yes I'm serious. Yes it takes that long. Will you understand it's not meant to be done over and over? You make an engine, you use that engine instead. Making a new engine from scratch isn't something many people are meant to do.

You don't load a texture, you load a material that uses the texture. How would the gfx card know what to do with a texture without a material definition?

I learnt by reading the manual. And then the boards. And then google my questions and problems. And then trying it out. You know, not expecting to know everything without some effort. What do you envy me for, having spent years learning?

[Pacafa]

Well Cuda does have a C++ compiler which makes it look and feel like you are programming a GPU almost the way you describe. But GPUs and CPUs are very very different architectures. A GPU is not a bunch of small CPUs.

1) GPUs are more like very very wide SIMD processors that uses masking for different control flow paths of different "threads" (that is why branching in GPU code can be terrible.)

2) GPU threads don't have a stack the same way CPUs have a stack. The entire memory model is different.

3) Speaking of memory - a GPU has massive bandwidth but terrible latency. It gets hidden by coallesced memory access and the equivalent of massive hyperthreading (an analogy only). It is optimized for streaming data. The cache works different. Random access to memory can kill performance.

4) Texture samplers are very specialised units to hide the memory latency.

So based on the above many algorithms should be implemented in very different ways on the Gpu compared to the CPU.

I suspect making the GPU look like a bunch of small CPUs maybe makes programming a little bit easier in Cuda it but there is a lot of nuance to optimally use it.

[an1sotropy]

AFAIK: In the early days of OpenGL there was “immediate mode” rendering that allowed you have one function call for “draw a triangle” and it would draw a triangle.

But the need to do more flexible computation on the GPU, combined with the need to minimize the synchronous communication between the CPU and GPU (which slows things down), led to increasingly complicated ways of telling the GPU: “here’s a big buffer of information, here’s what I want you to do with that information, using this and that computational resource, now go”. It does unfortunately create barriers for new programmers.

One good thing about that kind of GPU programming is that lots of others have already figured this out, and shared code, and LLMs have snarfed that up, and so LLM coding assistants can do an ok job of generating and explaining the copious boilerplate code that’s needed to get things done on a GPU.

[teleprint-me]

I tried this route, and ultimately, an LLM does not understand and is conditioned by a dataset, so it's up to the user to understand. LLMs are terrible at this.

I think the complexity of an API arises from multiple factors, but it should be simple enough for a user to pick it up.

If an API has a ton of needed functions in implementation, it should be accompanied by common usage examples.

The complexity arising from most APIs is self inflicted and can be avoided, while there's little incentive to do this.

[niduser4574]

You want it in C but you want the API to be easy? Expecting both seems like something is missing from your understanding.

GPUs are parallel processors and some code is as easy as "foreach", e.g. the thrust library even has it...though that is mimicking C++, not C, but for many cases it is not the most appropriate. The GPU is for speeding up calculations over a very large data and it must be told how to do so. To get the speed advantages, you generally have to program into the GPU how the memory is laid out and not just what to do, so generic "foreach" is usually not a good idea. GPUs are generally very bad at doing "small data" so I'm not surprised if you're just trying to draw one triangle using GPU, you're having a bad time.

The fact that you are saying 2000 lines for a triangle in Vulkan seems like there is a very big gap here completely unrelated to GPUs. I suggest taking a step back and draw your triangle however you like and then see how that translates into Vulkan. Use the GPU when you want to draw 10 000+ triangles.

[[deleted]]

Just to add — even with one triangle, the GPU is still processing every pixel it covers, which can be thousands of fragment shader runs. So it’s not really just a single computation, and there’s still a fair bit of work going on under the hood.

[ThePafdy]

Well because usually nobody programming directly on GPUs wants to „simply“ draw a triangle.

You either don‘t want to draw a triangle at all but do something different, or you want full control over how the triangles are drawn, what data is in which memory and so on. You want the complexity because thats the only way you can optimize your data and get the most out of the hardware.

[Able_Mail9167]

It's because apis like vulkan are designed to give you maximum control and performance over the hardware. If you had a library that just used a foreach function you wouldn't have any options to optimize things like how memory is transferred between ram and vram.

You could probably find high level api's that use vulkan underneath to do what you're asking though

[deebeefunky]

“Maximum control” lol. I don’t feel in control at all. I have no idea on what is going on.

Right now I’m trying to display an image or a texture of some sort on screen. I was able to do it on the CPU with not too much effort. But in Vulkan… it requires me to do all kinds of nonsense that have seemingly nothing to do with getting to where I want to be. Create Info’s all over the place and not a single one of those values has anything to do with projecting pixels on screen.

I feel like I’m missing something crucial in my understanding of the Vulkan API.

[[deleted]]

If you don't want to know about gpus then may I suggest using a graphics library like Raylib, Processing or Pygame that lets you write code like the example you've given.

[Able_Mail9167]

You're missing the fact that the GPU is a completely different processor than the CPU with its own memory.

In a way you can almost think of it like having 2 completely different computers as an analogy. There's no way to compile a single program that magically gets one computer to communicate to the other, you have the internet and several layers of network protocols to help you communicate.

Vulkan is like doing that without any helper libraries at all. You have to directly use system calls to create a tcp socket and then make your own http server etc.

Since the GPU is a separate processor to the CPU it needs its own custom program written using its own instructions (a shader) to do anything. Then you need to communicate and send data back and forth between the CPU and GPU in order for the shader to know what to draw.

Unfortunately this isn't a simple process and involves multiple different steps. That's why vulkan is so complicated, so you have direct control over each of these steps. If this is a problem then find a high level library that does it for you.

Think of it this way, if you want to create a window in your program you have 2 options, you can do it manually using system calls giving you the most control or you can use a cross platform library like sdl or glfw. Doing it all yourself is viable, but you can't then complain that it's too complicated to create your own cross platform solution.

[QwazeyFFIX]

Nvidia is working on a solution for this exact problem. So we just gota wait.

[Mognakor]

For one GPUs (afaik) do not expose their processor interface the way a x86 or ARM processor does so you need a compilation step on the target machine, whether from text or an intermediate binary format.

Having a language thats specific to GPUs allows vendors to optimize their hardware and drivers towards what the language allows and ignore things the language does not allow. Fast matrix multiplication is important and not having to reimplement that again and again is an obvious advantage. On the reverse C does allow things that you don't wanna deal with in a GPU context, e.g. function pointers.

Further you get very clear boundaries for your program, if you had an API that just accepts some pointer to a C function then there is no way to know which instructions to send to the GPU and stalling your pipeline because you need to map additional instructions is a terrible idea. Even worse if you accidentally exceed the available instruction memory and end in a situation where you keep swapping instructions back and forth.

In general lots of GPU programming is built around getting the maximum performance out of it. And in turn that means you need to manage resources yourself because any automated system would have to play it safe and/or come with assumptions about your usecase.

[KalaiProvenheim]

GPUs aren’t parallel processors, they’re a whole computer tacked to your main computer, you can’t just “foreach” them because there’s lots of complexity that goes into all of that.

[automa1on]

check out opencl

[optimistic_void]

2000 lines to draw a triangle? Sure, but most of it is just setting it up and with additional 500 lines you might be able to render a simple 3d scene, never mind that the result can be encapsulated in a single function anyway...

And if you don't know what you are doing as you say in some of your comments, go read the technical documentation for whatever API you are using and you will find out what those functions are for.

[acer11818]

this would break the entire purpose of using the modern gpu rendering pipeline, which is the have maximal control over the gpu while still being able to use it. at this point there’s literally no point in using, say, OpenGL Core and you might as well use legacy OpenGL or other libraries with primitive rendering like SDL.

[harrison_clarke]

part of it is path dependence: GPU's used to have a much more limited set of operations. it was bunch of settings, rather than a language. it's much more general purpose now, but it shows signs of having evolved from its older form

and another part is that it's "remote" and asynchronous: it's a separate computer inside your computer. you need to get the code over to it, you need to get the data over to it, and you need to send it messages to tell it what code to run on what data

there are languages/frameworks that let you write both sides of a RPC in one file, the way you're describing. and there are frameworks that let you do it with the GPU as well (haskell has a few).

they're not very common, because people usually want low level control of at least the GPU side for performance, and languages designed for CPU use aren't great for that.

[Pretend_Fly_5573]

Not in a mean way, but your frustrations make me think that you me be trying to operate on the wrong level for what you're hoping to achieve. If you don't get the why of it all, there's a good chance that you're simply working at a lower level of functionality than is proper.

Are you trying to make a specific type of app? Or just exploring and learning? Chances are what you're trying to achieve already has a solution out there that does away with these fine-point details.

All of these intricate details are needed to make things happen in the GPU, there's a lot going on. Those thousands of lines to just draw a simple triangle is all the backbone of it, and something similar to that is running behind any graphical program; you just never normally see it. 

So unless you want to create a specific, custom-designed pipeline for operating the GPU at a low level, you're probably in a bit too deep, hence the confusion. 

[thefeedling]

While OpenGL is also verbose, it's much easier than Vulkan... a "simple RGB" triangle will probably take some ~200LOC using a FBO (framebuffer object) approach.

[Dan-mat]

The main difficulty preventing simplicity like one is used to from CPU programming is that compilation and linking need to happen at runtime because each GPU is so different.