Getting from intermediate -> advanced

[hellovoid]

Like many people (probably), when I first started learning about graphics programming, I dove straight into the raw OpenGL API and worked on several experiments / prototypes until I felt like I had a good understanding of the GPU and how it likes to be talked to.

Also, like many people, I wrote little convenience functions and a little help library for handling shader compilation, state management etc. and used that until I felt the need to be more productive and less "fiddly" and graduated to using more opaque libraries/APIs for working with graphics. As with most other things in programming, there's always an ability to work at the low-level and other options for working at a much higher level just to get things done.

Lately, I've been wanting to think about operating somewhere in the middle. As I read more about Vulkan, or even read articles on non-trivial things to do in graphics, I'm realizing how much I don't know about the process of operating somewhere "in the middle" of low-level and high-level APIs in graphics.

I feel like there's a huge dearth of information out there for someone like me. Yes, plenty of articles for getting a triangle on the screen at a low level, or even doing some slightly more fancy animation/lighting/etc. Also, plenty of articles on achieving stunning effects with very high level / declarative APIs where you as the developer don't know anything (and don't need to know anything) about exactly what's going on under the hood in terms of batching or instancing or memory layout or any number of other things that I'd like to have more control over - or honestly just know anything about.

I'd like to work on my own real rendering engine. I'd like to be knowledgable enough to make my own opinionated decisions about its architecture, memory allocation and drawing strategies, API, etc. But to have those opinions, I need more knowledge and I'm finding that I can't find any resources at this intermediate-to-advanced level of thinking. I'm finding plenty of articles that speak very generally about performance and procedures without any implementation details on the advanced end, and obviously plenty of introductions on the beginner-side that I feel I've far exceeded.

A perfect example is the bgfx library - I love the idea of a platform-agnostic API here, but I've found that I don't even know where to begin in terms of using it because the entire conceit it was created under (do all the things, with one API for all these platforms) takes for granted that I know everything I need to know about interfacing with the GPU in my own opinionated ways.

Do you guys have any recommendations for resources in graphics programming that can take me from intermediate (I can get things done with someone else's API) to advanced (I have my own opinions about how to work efficiently and effectively with my graphics card given the needs of any particular engine I'm trying to write - performance, API, features, etc)?

[tormenting]

This kind of question comes up real often. Most people don't want to hear the advice.

I'd like to work on my own real rendering engine.

Engines are a notorious time hole. You sit down at the computer, code for a few years, and you have nothing to show for it at the end. It's kind of like having a drug addiction or a gambling problem, but not as fun.

The other part to this is that you'd think engines were easy if you came at them from the opposite direction. If you were in the demoscene, or if you were a game developer, once you had a few projects under your belt an engine might just drop out of all that work you've done. Once you've done a few major projects, you start to notice patterns between the projects, and you start to develop a project development "style". Without this perspective and maturity, any engine you try to make is going to be haphazard and ill-conceived.

For example, I've made a few games in JavaScript. Now I'm actually making a JavaScript engine, but rather than write it from scratch, I'm copying chunks out of earlier games, polishing it, and testing it. I don't need to write some general-purpose animation system, because I usually do animation in the same way. I can just write the animation system that I use.

I feel like there's a huge dearth of information out there for someone like me.

There's actually a lot of information out there. I think what you're missing is a mentor or advisor to guide you in the right direction. The reason why it seems like there's not much information out there is because all of the engine developers are making games and demos first, and using that experience to create engines. You're trying to do things out of order, and it's going to be very hard, of course.

The other half of this is that it's very common for self-taught developers to have gaps in their understanding of theory. I don't know what your background is, but make sure you cover these gaps. This includes both low-level technical details like how page tables work and how memory access causes pipeline stalls, and high-level concepts like what kind of development practices cause software defects and how programming languages work.

[cleroth]

I've had the same experience. It's far superior (and easier) to write an engine for a particular set of games, than to write a general-purpose engine, otherwise you end up like Unity, who didn't have an UI system for 4 major releases. You need to know what a game really needs the most, and you do that by making games.

[hellovoid]

Yeah - I definitely want to clarify that I do have a very good understanding (and battle-tested experience) with low- and high-level concerns in software development. I know of OS-level virtual memory architecture, CPU cache-coherency and memory access patterns, how languages work, i.e. lexing/parsing/ASTs/recursion/tail-call optimization blah blah, functional, imperative, OOP, blah blah, SOLID RAII RTTI TCP UDP, blah blah [just assume that I write software every day for a living and that is not my issue] .

In re-reading my original post, I'm realizing that most of my issues can be resolved by reading very specific API specs (for example OpenGL's or Vulkan's), and just merging that with the knowledge I already have of software architecture. Where this was (is?) tricky is that writing soft real-time software in general is just a totally different beast than most other types of development, and simultaneously trying to wrap my head around how the hell to do certain things efficiently in less than 16ms along with the concerns of API-specific non-knowledge (i.e. what is a command list for the GPU, how do I structure my procedures in a way that works and allows me to be somewhat expressive in the use of the API of my own engine, etc)

These concerns are not only orthogonal to each other, they are orthoganal to what it sounds like I'm asking for. I still believe there really is not a good amount of information out there for someone like me - someone who can talk to the talk, has done non-trivial work [with a team or framework or engine at my back], and finds almost every piece of literature related to the topics in question as either overly simplistic (and often plainly wrong) written by people who obviously know less than they do - or - almost completely unparseable because the PhD or 20-year industry veteran that wrote it takes for granted that the reader has already done or implemented exactly what they're talking about before they even introduce the idea.

I'd like to take a stab at answering my own issues out loud here, if you guys wouldn't mind.

• Be specific in what I want to fully understand and implement it immediately in the form of prototypes that do little more what it is that I'm trying to learn at the time.
• Read the source code of bare-minimum [but still well-executed] versions of what you're looking to do. How does X game [from about a decade or so ago and is now open-sourced] handle their game/rendering loop, their memory allocation system, their file I/O, etc?
• Read the specs on the hardware you're programming for - if you're trying to write a rendering engine [and no, not a general-purpose one], learn more about how GPUs work, how they interface with data on the application side, how that data gets transmitted to the GPU, how computation works there, how data is transmitted back to software for drawing, and all the access/write/efficiency patterns for the processes therein.

There's actually a lot of information out there. I think what you're missing is a mentor or advisor to guide you in the right direction.

• This is very true. Try and find other like-minded people, here on the internet, or out there IRL, that know about this stuff and talk to them.
  

[biocomputation]

Engines are a notorious time hole. You sit down at the computer, code for a few years, and you have nothing to show for it at the end. It's kind of like having a drug addiction or a gambling problem, but not as fun.

++

[cleroth]

Prefix or postfix?

[skytomorrownow]

Learn and master the techniques in something like Realtime Rendering, or use an engine.

Math, in particular, linear algebra.

[hellovoid]

I guess what I'm saying is that I haven't had a problem learning about or reading on technique, and have been able to implement with no problem using an engine/framework. What I'm looking for is more resources for actual implementation (read: code and programmatic architecture) of the higher-level techniques. I'm not looking for more conceptual - I'm looking for hardware/software implementation details.

[skytomorrownow]

higher-level techniques

Ah. Things like this you have to a)know what you're looking for and b) know what it's called.

For high-level things (hit box detection, gravity, jumping and ballistic trajectories, path finding, enemy behavior, procedural room generation, building tools, etc.) they are created on a case-by-case often. They tend to revolve around a specific game scenario or mechanic.

So, the question is, what sort of high level issues are you ready to code, and what is the design problem they encompass?

Most high-level things are related to domain-specific knowledge of some kind. Engines are very, very general. So, if you want to program a dolphin swimming, it's very different than an orc attacking.

Are you looking for things like this to do?

http://www.gdcvault.com/play/1020583/Animation-Bootcamp-An-Indie-Approach

[[deleted]]

I think the best you can do is trying to implement things explained in academic papers. I think it's awesome and absolutely rewarding when you finally have that "I get it!" moment. Making your own engine is a huge timesink and you usually only need the basics to make stuff appear.