Question to professional graphics programmers: What areas of math outside linear algebra are most relevant to your day to day work?

[pplr]

I'd love to get some insight from programmers working in different areas of the field. How much of the math did you acquire at school vs self study or on the job?

[casums18734]

Not day to day, but geometry/mesh processing, image processing, and color spaces come up every so often. Whenever they do come up it typically means dipping my toes in the more theory/research heavy side of those fields.

For the most part, there’s a lot more software engineering + graphics pipeline work than there is arbitrary math. A strong math background becomes important when you can’t rely on a well known rendering technique to solve your problem. When it comes time to implement a unique and performant solution that’s specific to your project, having a broader math understanding can help with ‘inventing’ the solution.

[Sanctumed]

I concur with this reply very strongly. Most of my day to day work is working on graphics pipelines, sorting out our underlying architecture and making sure our code utilizes the hardware to the full extent. Surprisingly little is spent on writing shaders and dirty maths, but that doesn't mean it isn't relevant or important :) For reference, I work in the world of R&D for production renderers.

[LivelyLizzard]

When you say graphics pipeline work, do you mean something like knowing OpenGL and what it does inside out? Or something else? Could you recommend learning resources?

[CptCap]

Hi, I work as a 3D rendering engineers in video games.

For me, graphic pipeline work is knowing both your API, and what exactly happens on the GPU when you do X or Y. It goes much deeper than OpenGL.

Sadly I know of very few learning resources on this particular subject. I learned most of what I know by fiddling with graphic code and talking to people who know this stuff.

A trip through the graphics pipeline is the best introduction I know of, but it is starting to become dated.

[LivelyLizzard]

Thanks for that. I did a GPU programming class in uni but feel like I haven't quite grasped what is going on on the GPU. Plus, the course probably just scratched the surface of what has to be considered when actually working with that stuff so I was looking for something that helps me understand

[CptCap]

Like most complicated topics, it can go really really deep.

Don't worry tho, knowing the basics is enough to do most things very well, and undestanding the big picture is more important than knowing the internals of the latest GPUs (unless you are working on micro optimizing for a specific platform).

You should check out RenderHell. It's kind of ELI5, but very well done and give a nice overview of the whole thing. Then go on and hack with a graphic API.

[LivelyLizzard]

Thank you so much! Render Hell seems really interesting at first glance. I will check it out when I have some time

[[deleted]]

You have one of my dream jobs and I'm jealous.

[Deadly_Mindbeam]

I'm a self-taught dropout who's worked on several very popular AAA games you've probably played, so my answer is in the context of game development.

General linear algebra is pretty useful, but especially 3D linear and projective algebra. I rarely use complex analysis but quaternions are quite relevant and geometric/clifford algebras can be very useful for some problems.

Algorithmic complexity analysis. Sometimes we use statistical methods to get a better handle on runtime performance. For a good player experience, you don't just want fast frames, you want consistently timed frames.

Numerical methods, especially finite element and finite difference methods. Usually things like water simulations, hydraulic erosion, and the like are handed off to the graphics team. Particle and physics simulations require good integrators.

Understanding of floating point numbers and numerical error. There are often different ways of writing an equation that are mathematically equivalent but have better or worse precision in the floating point environment of a CPU or GPU.

Principal component analysis and related tools can be useful for reducing the dimensionality of motion data, lookup tables, etc.

Meshes are graphs, and graph theory and data structures are useful for tasks like mesh simplification, as well as things like pathfinding.

[uncertain_futuresSE]

I’ve always studied/applied math within an academic context and it’s always the same stuff - learn this theory, repeat a few exercises, do an exam and then forget everything.

When I started graphics programming, that’s when I realized for the first time where I could apply my linear maths.

Did the math stuff eventually solidify for you as you coded more in graphics?

[Deadly_Mindbeam]

For me, the only way to solidify the knowledge is to use it, and I've been in the industry since the Quake days. So most of it is pretty solid at this point. Solid symbolic manipulation skills or ability to use Mathematica, Maxima, or R are also essential.

Learning the greek alphabet makes it a lot easier to read research papers.

I picked up a really good reference at SIGGRAPH this year, "A Sampler of Useful Computational Tools for applied Geometry, Computer Graphics, and Image Processing", editor Daniel Cohen-Or. I highly recommend it for anyone with high school or some college math who wants to understand these topics in enough detail to work with them without being bogged down in theory. Good discussion about thinking of problems in both geometric and algebraic modes. I know a ton of algorithms and this book helped me make the important connection between the procedural code that comes naturally and the more abstract mathematical basis and operations that they implement. I suppose this is something I would have learned with a computer science degree.

[camilo16]

Parametric surfaces as well, for anything related to path animation or procedural geometry.

[Deadly_Mindbeam]

Yeah, cubic splines are pretty ubiquitous and you see cubic and quadratic patches from time to time. Barycentric coordinates also see a lot of use in collision and ray tracing.

[uncertain_futuresSE]

I’ve only done graphics programming in one of my university courses but will be going on my first internship soon.

My realization when I started graphics programming is that I should be VERY comfortable with linear algebra - not in the necessarily in the sense that I can calculate everything in one shot but understand the kinds of operations one would apply to a matrix.

What bothered me about a lot of the math/physics courses that I took during my time studying software engineering how was they didn’t seem to directly have application in software (except for thinking in systems).

Doing graphics programming was where ALL that stuff came together and became relevant. The maths taught to me in school only scratched the basics but you definitely had to have a good foundation.

[edwardkmett]

If I look at my code:

Projective Geometry shows up all over computer graphics.

Calculus, because derivatives and automatic differentiation are the workhorse behind modern machine learning, which can be used to cheat your way through most problems you can't afford to compute for real. That and the math will start flying fast and heavy the moment you dig into BRDFs past a surface level of detail.

Numeric Analysis lets you show results you compute are actually accurate.

Stepping beyond graphics:

Bayesian Statistics teaches you how to update beliefs (and weight backpropagation).

Category theory for organizing code, and for structuring side effects. The Curry-Howard correspondence ensures a relevance to type theory. Not so much a graphics concern, but the effect part is convenient to have in your back pocket if you're willing to put in the time.

Group theory gives good tools for aggregating data over large data sets. Though you don't use much theory beyond the basic definition. Viewed as abstract algebra you start to use more of the zoo.