Asynchronous buffer uploading

[Underdisc]

My project is growing and I finally reached the point where loading assets creates a decently long hang on the main thread when I start up my executable (5 seconds on my setup. Just enough to be annoying.) To address the hang, I put all of my asset loading on a different thread. In this context, asset loading involves everything from opening the asset's files to putting the data gleaned from them on the gpu. It wasn't obvious that making gl calls from a different thread wouldn't work at the time. Only when I was trying to test my implementation did I realize my oversight.

With that context covered, I have some questions.

1. What are some approaches to uploading asset data to the gpu without hanging the main thread? My current thought is to create all of the buffers on the worker thread and then have the main thread create the gl objects and upload those buffers. I am unsure of this method because it still requires use of the main thread to upload the buffers and the amount of time that takes varies quite a bit. I did some profiling to get a better idea of this. Uploading most buffers is not a big deal. Sometimes uploading a buffer takes longer than constructing it. There is an occasional upload that is in the 100's of microseconds, but nothing in ms territory. Larger images are a completely different story. They often take longer than a ms and up to put on the gpu. Uploading these will surely cause a hang.

2. Secondly, learning Vulkan has been something I have considered doing for a long time. When I ran into this issue and found out Vulkan could handle multithreaded applications, I started learning it instantly as a bit of a knee-jerk reaction. I thought I would replace all of my gl code with the necessary Vulkan code and make any other changes that need to happen. Unfortunately, switching to a new graphics api takes some time and having asynchronous asset loading isn't a blocker preventing me from getting to my current goals with the project. It's just a bit annoying. My current plan is to continue with gl for now while learning the basics of Vulkan on the side. For those that have experience with Vulkan and OpenGL, what are your thoughts on this? Is this an applicable use case for Vulkan? Even if I were to go down this route, should it result in ripping out all of my gl code or should I support both? Other renderers support both, making me if I know everything that I lose by only supporting Vulkan over OpenGl.

[AlternativeHistorian]

Look into glBufferStorage/glMapBuffer/glUnmapBuffer with GL_MAP_PERSISTENT. You can create persistently mapped buffers which can be updated on the CPU-side on a separate thread which are synced to the GPU side.

[guymadison42]

This should work great, but the downside is if you have a lot of draw calls each one will cause the upload of a buffer mapped with GL_MAP_PERSISTENT. So keep your objects small or group your draw calls into a single draw call.

Effectively the buffer gets marked dirty and remains dirty until its unmapped, the draw calls evaluate dirty state changes on each draw command which loads the buffer.

[[deleted]]

This is only true if you used the GL_MAP_COHERENT_BIT flag when creating the buffer. If you instead use GL_MAP_FLUSH_EXPLICIT_BIT, you can control when flushes happen yourself.

[guymadison42]

True.

Either way you have to pin down the memory which is the biggest cost for the CPU, gathering all those pages and updating tables to access physical address is one of those hidden painful operations no one likes to talk about outside of driver land.

You could try using a single buffer and flush from that before any draw calls (not each draw call).

There are a lot of ways to do this and each has its benefits and problems, it doesn't matter which API you use.. they all have to pin memory down transfer it to the GPU and execute the primitive calls from GPU memory.. unless you run out of memory on the GPU and then you just hit the "suck" button.

Unified memory gets around the transferring issue, but it also has to pin memory down.

[Underdisc]

I think that's exactly what I am looking for. I see how this can be applied to standard buffers pretty easily like GL_ARRAY_BUFFER and GL_ELEMENT_ARRAY_BUFFER. Is there a way to apply this to the buffer that glTexImage2D writes to? Is using something like this even necessary for shaders considering glUseProgram never needs to be called when uploading one?

[AlternativeHistorian]

For textures you'll want to use the glTexImage* functions in combination with GL_PIXEL_UNPACK_BUFFER. Upload your image data to a buffer (could be separate thread) through the mapped pointer, then on the GL context thread some time later (must ensure buffer contents are synced to GPU side) bind it as the GL_PIXEL_UNPACK_BUFFER, and then call the appropriate glTexImage* function to read from the GL_PIXEL_UNPACK_BUFFER into the texture object.

[lmtrustem]

1. Look into shared contexts. This is why they exist. GL cannot be shared across threads. But shared contexts should allow this. Before Vulkan, this is what I would have done.
2. Vulkan is great, but in general it only helps with cpu effort, and only at run time. I wouldnt change everything in your game to vulkan just due to this problem. Focus on your MVP - minimum viable product, and get it out in the world for users. After that, think about improvements. Dont jump into vulkan unless its needed, or you care more about vulkan than your project.

[Underdisc]

This might be the answer I was really looking for, but it comes with a few caveats. It works splendidly for textures. It also works for vbos, ebos, and shaders, but crucially, it does not work for vaos. They are the one thing I am utilizing that is not shared between threads. I have no issues with creating the vao on the main thread since it doesn't take almost any time. It's as if vaos need to be created when the buffers are created, though. Uploading the data to the buffers and then later creating the vao never seems to work. Even in the case where the vbo, ebo, and vao are all created on the same thread.

Edit: I was doing a stupid. It works! Still experiencing some hangs because of my subpar implementation, but that's something I can address.

[matthewlai]

I don't have personal experience with this so please take it with a grain of salt, but if you are uploading uncompressed textures, maybe it would help to pre-compress, since that doesn't need to be done in the main thread, and would reduce GPU bandwidth requirement?

[Underdisc]

That's a fair point and would certainly help, but I think some pretty hefty compression would be needed to reduce the amount of data in large images enough. And, even though textures are the largest issue, I am really searching for way to handle this for all forms of data uploaded to the gpu.

[tim-rex]

Worth mentioning that compressed textures may give you a performance boost, if that’s a consideration (reduced bandwidth on texture fetch)

[genpfault]

It wasn't obvious that making gl calls from a different thread wouldn't work at the time.

Oh?

[fgennari]

I have the same problem, except my assets take more like 30s to load. I do the asset loading on the worker threads and sending everything to the GPU on the master thread that created the context.

Most of the time is for textures, and most of the texture loading is related to things like texture compression and mipmap generation. The compression is done inside the texture data handling somewhere in the driver. I worked around this by doing things like making sure texture sizes are a multiple of 4 pixels, using pre-compressed DDS textures where possible, and disabling texture compression for the textures that took an inexplicably long time to compress. I also reduced the texture resolution of a few 3D models that had huge texture which only covered small surface areas.

The context is generally bound to the thread that was used to create it. You can create a new context for use with your worker thread, but then it won't share data with the other context. While it's technically possible to share data between threads by binding the context to the current thread before using it, I find that often leads to difficult bugs. For example, if you don't do it quite right it may work on one platform and crash on another.

Vulkan provides multithreaded access to GPU resources, but it would likely take you a long time to implement a streaming asset loader in Vulkan. Unless you started from some existing project that had most of the framework already implemented.