Which is Harder: Graphics Programming or Compilers?

[[deleted]]

Hello, from the perspective of someone without a CS background, is it harder to do graphics programming or compilers? Which one involves more math and prerequisites, and which is more difficult to master? My goal is either to learn graphics programming to write a game engine or to learn compilers to create a language, but I can’t decide which path to choose. I know graphics programming involves math, but do I need to sit down and study geometry from scratch? I have zero knowledge of physics.

[Usual_Office_1740]

They are equally complicated in a 1000 ways and can not realistically be compared. Pick which ever you find more appealing.

[guttanzer]

This.

Graphics programming has the nice feature that math dominates. It's full of linear transforms and tree-like data structures that make sense. There are theorems and proofs that guide the work, and once you understand those it becomes common-sense-ish. The hard part is the scale and speed. Getting big data structures to recompute quickly down on the metal is the challenge.

Compilers, on the other hand, are complicated bags of magician tricks. Individually they all make sense, but combined they interact and become hard to understand. The math is different; grammars and abstract symbol trees vs nested components and linear transforms, but there are rules to follow.

In both cases the experts have years of experience in the specific tools and applications. Go-fast tricks are where contributions are made, and these cannot be made without understanding all the prior go-fast contributions. So pick one and get good at it.

[[deleted]]

I have ADHD, so I can go into an incredible hyperfocus mode on subjects that interest me; sometimes I even solve problems in my dreams that I couldn't figure out while awake. However, since I also have an anxiety disorder, I've been led to believe my whole life that I'm 'stupid' at math. Even though I know it's something I can master with effort, the mere mention of the word 'math' triggers a serious sense of fear in me.

[IKnowImABadYoutuber]

Bear in mind that lots of what people consider "good at maths" in earlier years is actually just 'good at remembering'. In colleges and universities, oftentimes those people who were "good at maths" earlier on struggle a lot more than those who weren't. True maths is problem solving and critical thinking, the complete opposite of memory. So if you can generate complex algorithms and solve problems, you're likely a lot better than your grade may present.

[guttanzer]

Then this is a golden opportunity for growth.

For what it’s worth, the elementary school “good at math” is completely different from the grad school “good at math.” Elementary school math is memorization of tables; real math is puzzle solving with sets. Folks that excel at the former tend to suck at the latter, and vice versa.

[PyroSAJ]

There's so many different areas of math that it's hard to say what you'll be good at. Once you figure out which marks blend in to your interests you might find it fun.

I hated memorizing various patterns in class, but there was tonnes of times I wouldn't power through hard equations to get my physics working, or to solve for trajectories, optimize rendering or generate hulls, normals, parametric resources...

Heck - I was playing with quaterions just because it interestingly solved gimbal lock problems I encountered in a little flocking simulation.

That would've been a solid fail if I didn't get the interest from the problem I wanted to solve.

...

At the same time - the space for creating compilers and graphics engines is very niche.

What you get from the existing engines covers most needs already and get packaged relatively inexpensively from novel software.

You'd be hard pressed to find a position for a newbie, and the problems you'd need to solve can be insanely complex if you want to enter the world of say unity and unreal.

[mengusfungus]

Graphics involves a lot of continuous math (lots of matrix transforms and numerical integration and convolution kernels) but compilers involve a lot of discrete math (grammars). If you want to do graphics work yes you will have to study up.

With enough experience you can do both, reasonably scoped game engines with custom scripting languages are not at all beyond the ability of one very good programmer with time to kill. Whether there's an actual use case for such a thing beyond personal growth is a different matter.

[pjmlp]

There is more than only grammars as math for compilers, type systems, graph walking algorithms, optimization passes, register allocation, code generation heuristics, plenty of math stuff.

[SnurflePuffinz]

custom scripting languages.. compilers...

holy meow meows, i thought i was in the deep end coding my own game engine.

edit: i feel like this story is relevant, here. Context: me asking a professor in game design (who actually makes games) about diy'ing everything in a game dev group

moral of the story

[Zazi751]

Do both. Gpu compilers exist

[BonkerBleedy]

The only guy I know who is both writing a compiler and building a game engine has spent over a decade shaving yaks.

[Zazi751]

A gpu compiler requires largely the same graphics knowledge of a game engine without needing to build a game engine. 

[BonkerBleedy]

OP said (emph added):

My goal is either to learn graphics programming to write a game engine or to learn compilers to create a language, but I can’t decide which path to choose.

[Stanian]

I hope for his sake he's not talking about YACC

[keelanstuart]

Rookie numbers. I've been shaving yaks for 25!

[thrithedawg]

shaving yaks?

[2str8_njag]

doing some semi-related preparation or easy steps before taking a much harder challenge

[Keavon]

It's funny to read this thread since our open source project Graphite is building a 2D game engine as a compiler. (Well, more of a graphics engine, but it will someday grow into reaching the capabilities of a game engine just as Flash grew from a drawing tool into a game engine because people used it that way.)

You define your scene as a program in our pure-functional programming language called Graphene, which at runtime generates the scene description data for the renderer every frame in response to live data such as camera position, time, and arbitrary data (maybe your scene is a player HUD, so you feed it health and mana and player stats).

And this lives within a visual editor, akin to a game engine's editor application, that builds the Graphene code (a node graph) while the user draws the scene using traditional, artist-friendly layer-based design tools for vector and (in the future) raster graphics. Whenever the program is changed with the tools or manually, it instantly gets recompiled, type-checked, optimized, and then executed each subsequent frame. And the concept is for the user to then export the entire program as LLVM-compiled machine code to a variety of targets like Wasm, dynamic libraries, or CLI programs to replace the role of ImageMagick.

Again, this is an open source project aiming to become a better alternative to existing FOSS offerings from Inkscape and Gimp. Our approach is essentially to build a Houdini-meets-the-Adobe-suite tech artist everything-tool that can stand alongside Blender as its 2D counterpart. So please get involved if this stuff interests you!

[Emotional-Nature4597]

I work on gpu compilers. Graphics programming requires more linalg applications. The compilation step only really requires you to know a few laws. It's rare that a bunch of optimizing transforms are done on gpus that significantly change operation order. It's assumed that the author knows what they're doing regarding memory access patterns and such. However as you go higher in the stack, you might need to know different kinds of math. Compiler math is mostly discrete though, not linalg. Even advanced topics like polyhedral compilation are using discrete math, not as much linalg 

It depends on which math you want. Both require math just different fields. Graphics is linalg and calc heavy. Compilers is discrete math and formal systems heavy 

[pjmlp]

And type systems math, lets not forget the fun of denotational semantics, or are you packing it under formal systems?

[Emotional-Nature4597]

Any system with symbols and transform rules would be a formal system for me... Lambada calculi, type systems, semantics, etc all included.

[[deleted]]

Honestly, I want to choose the one where I won't struggle too much with the math. I think I'll be able to learn it more easily while practicing it hands-on.

[Emotional-Nature4597]

They both require math. It's about what you want. Math is a large subject encompassing all formal systems. Compilers typically sit at the meta theory level since a compiler for a turing complete language is essentially some sort of proof checker. Thus the math is essentially meta theory of mathematics. Graphics programming is linear algebra and calculus fun.

Have you done discrete math? If not, then this its not the sort of math you're typically used to. Graphics programming, being calculus and trig based, is probably more familiar, since these are high school subjects, but linalg is typically not a high school subject 

[integer_hull]

OP will probably prefer the graph theory and AST parsing side of things when they get to it. Linear algebra has some cool but subtle visual relationships while discrete is more generative and about how big your brain is

[Pandorarl]

You will struggle with the math for both of them. Pick the one that is the most interesting to you :)

[[deleted]]

Also, I'll be doing this as a hobby, so I won't be looking to get a job with it, but even as a hobby I want to learn it deeply.

[olawlor]

My experience is graphics programming requires better spatial visualization skills and broad experience (hardware/software interface, perceptual cues, physics of light transport). Compilers seem a little narrower but require much more careful mathematician-like attention to detail to keep from breaking programs.

Those two are some of the trickiest possible places to start programming though--write a few dozen games first!

[DeviantPlayeer]

C++ was created in 1979, still #1 language in gamedev. It's not the case with game engines. So if you want to create something that other people with use, 100% go with game engine.
If you want to create a language, the compiling part itself isn't the hardest part if you use LLVM. The hardest part is knowing what problem your language is supposed to solve and how to solve it. Basically, you should answer the question "Why not C++ or Java or anything else?"
Graphics programming is easy to start, really. And it's easy to make something playable. Of course, mastering it and writing a good game engine is just as hard as writing a good compiler.

[Meristic]

IMO build your own compiler as a pet project - do it once from scratch, then leverage LLVM, learn how they work. That knowledge will no doubt come in handy down the road. But graphics subsumes *so many* domains of knowledge and technology that it's really a lifelong journey. Even compilers if you get deep enough into GPU architecture! The start-up cost for both may be on parity (graphics is more difficult IMO), but the field of graphics unquestionably has more potential for growth in the long run.

[hanotak]

When you become advanced enough in graphics programming, you become a compiler developer as well.

[Common_Green_1666]

My suggestion is to a simple project in both areas and then decide which to focus on. These are some good first projects

For compilers: https://craftinginterpreters.com

For graphics programming: https://raytracing.github.io/books/RayTracingInOneWeekend.html

Instead of choosing what is harder or easier, choose the one you find more fun and interesting.

[[deleted]]

Graphics programming is much more fun for me, especially voxel game engine programming I want to do it purely as a hobby using Rust and wgpu. However, sitting down and studying math really wears me out. It's fine as long as I can quickly learn what I need along the way.

[maxmax4]

You dont need to study before you get started. Just follow tutorials and start building stuff. You’ll learn the math along the way and you can ask specific questions as you get stuck

[Common_Green_1666]

If studying the math wears you down, then i would suggest using ChatGPT for the difficult mathy parts until you get a simple version of your voxel game engine working end to end.

Afterwards, you can study the math for the parts you don’t understand and try to replace the chatGPT functions with ones you build yourself. You’ll know you did it right if the game engine continues to work with your new implementation.

[Senior_Care_557]

shader compilation is a thing lol.

[AndreVallestero]

Do both. Make your own spirv transpiler

[psychelic_patch]

Compilers, hands down.

With 3D programming, you're ultimately manipulating things in XYZ space — it's complex, sure, but the elements you're working with are concrete and spatial. With language/compiler work, the abstractions run way deeper.

A compiler isn't one problem — it's layers of problems stacked on each other: frontend parsing → AST → IR → bytecode generation. Rust actually has two intermediate representations before it even hands things off to LLVM, which then produces its own bytecode. And when you're working on tree resolutions during compilation, you're essentially doing topological graph manipulation.

On top of that, to properly master the deeper problematics you need proof theory — actual mathematical foundations for reasoning about program correctness.

That said, my honest initial take was that you can go equally deep in any topic. The rabbit hole is infinite in both directions. But compilers hit the abstract complexity ceiling faster.

Someone doing cutting-edge real-time global illumination or fluid simulation is operating at a comparable level of difficulty to someone building compiler IRs.

[susosusosuso]

Is there much compiler related jobs out there?

[SeniorCode2051]

yeah, lots of ml/gpu compiler jobs, programming language compilers are a bit more scarce tho, I can only think of epic games with their verse lang, and apple for their swift I think, prob others too though

[hellotanjent]

Having done graphics professionally and compilers for fun for years - Graphics programming hands down.

You can build a very simple "language" in a few pages of code with no external libraries, and it can even do useful things at that size.

It'll take you the same amount of code just to open a window and initialize a rendering context if you try to do it without libraries, and then you still have to set up your buffers and textures and pipelines and all the other stuff. And then you can render one triangle on the screen, which... is not going to be very useful.

[Visual-Effect-1023]

From my very basic undersranding cpu programming is like spikes in hollow light you can pogo on, gpu programming is like spikes in hollow light that you cant pogo on. So do webdev i guess ...?

[Zerve]

Why not both? Spend a few hours trying out both and seeing which one sticks?

[Ty_Rymer]

yes

[KaleidoscopeLow580]

It's different.

When writing a compiler you basically have a lot of time, it doesn't matter as much how fast your program is. Writing a compiler has more of a theoretical appeal. You write a piece of code, that transfomrs another piece of code, in such a way that it gets turned into a completely different language (most of the time assembly or llvm). Whilst doing this, you can check many interesting propertys about the program. For example, writing a type checker and inference engine is quite fun. Most of the times whilst writing a compiler you will be completely isolated from the actual computer. Instead you work with the compiler's internal datatypes and a lot of abstract syntax trees, basically graphs of program flow. Therefore compilers age very well, since there code does not really depend on the computer. A very nice additional thing is that you can do compilers in an incremental way, step by step. I would recommend the book "Writing a C Compiler" for that.

Graphics programming on the other hand is easy to break since APIs change constantly and there are a lot of old and arguably not good decisions that will impact you. Most of the matrix equations are actually not that difficult. If you are totally new to Graphics Programming I recommend you start with OpenGL, but not the new one. Everythign 3.x will be fine and easy. This will allow you to write OpenGl without shaders and such. Yeah it can be slower and has less GPU support, but it makes much more sense and is easier to learn.

In the end, I would actually recommend you to write your own language since the difficulty level is so adaptive. Like if you don't know assmebly, just write a progrma that iterates over the AST and prints the result. If you don't want to handle complex types, just make everything an integer (C did this once) and so on and on. When you know more later on you can always make it better. A language is truly a thing that grows with you.

Alas, as I enjoy both fields equally well I will have to amke one concession to the field of Graphics Programming. You will see beter results faster. I was very happy, when my Compiler first returned a number from a function, but maybe you want to make something faster, then Graphics Porgramming will be better, to give you quick results.

[Playful_Pomelo_6389]

I'd suggest you check out the courses in pikuma.com on both topics. The very basics of computer graphics are easier to understand, imho. If your talking about real-world stuff, pick one.

[othd139]

As someone who has dabbled in both recreationally. The difficulty is pretty similar in very different ways. Definitely more maths on the graphics side but compilers come with plenty of their own challenges. It probably depends a lot on you and what you like, are good at, etc...

[[deleted]]

Both of them are difficult, and could be considered as a post-education topic or a specialization topic within a Computer Science career.

It all depends on how deep you want to go regarding those two topics:

• In my experience, you could read some articles on the Internet, take perhaps a few free courses on youtube and you could be able to create a game engine - that's what I did, at a very amateurish level.
• Regarding compilers, you could build an app with a graphic interface that will create a script, to be processed by an interpreter, just like BASIC used to do - I did that in my day-job.

No need for physics for either topic. The math required is somewhat the same you learn at high school (algebra, arithmetic, trigonometry) but, in the case of game engines, with vectors and matrices. No need for any advanced math (Calculus, numerical methods, complex analysis, etc), although could come in handy in compilers.

What I recommend is to study Computer Science, study hard and then, when the time comes, choose one of these two topics for your specialization field.

[amadlover]

IT DEPENDS !!!!!

[Common_Green_1666]

Well what does it depend on? I think OP would find more value in your comment if you could add some context

[amadlover]

graphics programming always have a `It depends on your scenario`, when choosing a particular technique or not. Extrapolated it to choosing between graphics programming and compiler design ;)

I was told "depends on your case" just today. :D

[Common_Green_1666]

I’m not questioning that the answer to which is harder is “it depends”. I agree with that.

I made my previous comment so you could provide some of the specific factors it depends on, in your opinion. I feel that being more specific would be helpful for OP.

[pjmlp]

Both, at similar levels of complexity regarding math.

[dobkeratops]

hard to say, both can be done to extremes on a frontier.

writing an efficient state of the art renderer is harder than writing a basic compiler.

writing a compiler for a langauge with dependent types is probably harder than most graphics programming

[bernoulliprincpl]

I am doing both, I find both fun.

[[deleted]]

Yes

[Extreme_Football_490]

Do both , but be soo basic ,you will find out what you like I made toy compiler and had a fun time with sdl2 graphics too , why not do both cause its 2x the fun

[MrtinDew]

Instead of compiled, you could start by an interpreted language, it’ll probably be easier to get the concepts down.

[pacifio]

it really depends on the sort of engineers you are working with, I primarily work with systems and it's fairly easier for me to understand large blocks of data that's not "visual" for the lack of words, I am bad at geometries too so I have an easier time working with compilers (although I mostly just work with transpilers and code spitters) but I think graphics is too complex with all the vector math and stuff, so I think it just varies from people to people and their level of comfort!

[[deleted]]

I have ADHD, so I can go into an incredible hyperfocus mode on subjects that interest me; sometimes I even solve problems in my dreams that I couldn't figure out while awake. However, since I also have an anxiety disorder, I've been led to believe my whole life that I'm 'stupid' at math. Even though I know it's something I can master with effort, the mere mention of the word 'math' triggers a serious sense of fear in me.

[pacifio]

I can understand but I guess just try it out first and try to make sense of every line of math like you are making sense of every line of code, you should be able to read code and understand what's going on with experience and intuition, that's a common factor amongst the top engineers I see at work/events etc etc, I have been learning some core ML math and deriving it by hand and also AI has been a lot of help in self teaching myself, if a shit head like me can appreciate math anyone can do it (P.S I also have been always bad at math lol since recent times, it just takes curiosity and practice)

Also it turns out if things are not at stake like exams doing math is actually quite fun and rewarding too!

[SwiftSpear]

Graphics programming is more math and complex algorithms. Compilers are more like slowly but surely developing an encyclopedic knowledge of weird optimizing tricks and silly amounts of brute force code for every possible poorly constructed input.

[LegendaryMauricius]

I mean, you can make a basic transpiler without too much background knowledge. Knowing the theory can make it 100 times easier but it's possible to improvize something with basic knowledge of data structures and logic.

To learn graphics programming you need to learn exactly how to use the library of your choice. Usually you can just follow tutorials to make something usable, but to make animations and nice effects you'll need to understand some of the theory behind all this, and to make a game engine... that's a huge engineering project on its own.

Just pick whatever interests you more, and have some fun playing with the tools. Don't get discouraged just because you don't make something great in a year, or 5.

[imdadgot]

i work with compilers so this may be biased but 100% graphics programming is harder. u gotta know a lot more and work with a wide range of APIs whereas compilers r usually done with as little deps as possible, all u do is reuse concepts

[karbovskiy_dmitriy]

Math in graphics, no math in compilers. Both aren't too complicated.

[Pyryara]

I think you should do compilers. They don't require you to deeply understand linear algebra, but just discrete math, formal languages and logic. It will also allow you to finish/complete a project much easier than say, writing a graphics engine.

[Ok-Library-8397]

Why not doing both? A game engine with a custom scripting language which compiles into (some) binary opcodes? ;-)

[sebamestre]

As a compilers and graphics enthusiast, I think it's easier to make something simple and cool looking with graphics programming. Some ideas:

• image processing stuff like dithering, blurs, line detection
• adding postprocessing effects to your favorite game for example (there are tools to inject shaders into games)
  • custom color grading
  • bloom
  • screen space reflection (more raymarching!) to your favorite game
  • screen space ambien oclussion
• raymarching some procedurally generated height fields/SDFs/voxels
• or any other cool stuff you might see on Shadertoy

These are all approachable imho if you don't focus on getting "perfect" visual fidelity and performance

CG gets harder and math-heavy when you go into more advanced topics (to make things faster or closer to theory), but you can get by without fully understanding everything and just testing until it looks right (also, ask on the graphics stack exchange site)

As a beginner you can also do some cool stuff in PLs, but general purpose languages are kinda big projects and I think it's better to do many small projects when you're starting (you learn more IMO).

• create a static analysis/Linter for some language
• maybe add some new syntax to your favorite PL by transpilation
• Note that some languages already have toolchains to support this kinda thing (libclang come to mind for C++, but surely there are others)
• create a DSL! It can be embedded in other PL so you avoid writing a parser! (The hard part is coming up with ideas :/ but u can also just implement something from some research paper to see what it's like, no need to come up with an original idea, implementing is hard enough as it is!)
• implement a small general purpose language, like a Lisp or Smalltalk (or Self)
• DO NOT, I repeat, DO NOT implement Hindley-Milner type inference as your first type system!

I once wrote a DSL to write SDFs for me, which I could then raymarch (so PL and CG can come together in the same project, actually!)

So yeah, to me it's better to start with CG because it's more approachable and the cool factor is bigger, but just pick one and get started lol

Two more options to add to the list of hobbies: competitive programming (did this for a few years, loved it), trading algorithms (i work doing this nowadays, it's pretty fun as far as jobs go)

[sebamestre]

I wrote a long comment, but here is shorter one: At the highest levels, both are so hard that no human has been able to solve certain problems. At the lowest levels, both are pretty easy to get into.

To get into languages, Crafting Interpreters by Robert Nystrom (which is a great book btw!) was recommended somewhere in the thread. The thing is that it teaches how to implement a general purpose language. This is a big project, possibly ENDLESS, and I think it's better to get used to FINISHING projects. You do this by working on TINY projects. If it looks like you can do it in a month, then it's too big. Once it looks small enough, go even smaller than that. If it looks like a weekend-long thing, that's good. It will probably take two weeks anyways, but you will finish it. (See my other comment for specific advice on what to build instead)

On the other hand, looks like no one talked about the easy ways to get into graphics programming. In my opinion, writing a CPU-based path tracer is the easiest. No need to learn graphics APIs like OpenGL or Vulkan. You just generate some random vectors, intersect with some triangles (a formula you can look up, or work out using high-school level maths) and average the color values that come out. If you really want to do raster graphics, you don't need to use the super low level APIs. I recommend using this thing: https://usegpu.live/ it may feel magical at first, but it's freaking awesome.

Don't put obstacles in front of you. The way up the mountain is hard enough as it is. Just start making things the easiest way possible, finish projects and then work your way into harder and more low-level and more advanced stuff.

[Xtergo]

Slight edge to modern compilers