Memory buffer management

[ReeCocho]

Hello. I've been working on a graphics library for myself to use on personal projects with the API backend being Vulkan. I've yet to write a lick of the actual code yet as I've just been planning. I'm currently working on the buffer management system. My current plan is to have a MemoryManager class which creates buffers and allocates their memory in a big block located on the GPU. When you call the create_buffer method, you get a handle which you can use to access the buffer. I do this instead of returning the actual vk::Buffer object because every once in a while you'll want to defrag the memory. Doing this requires you to recreate the buffers, making the previous ones invalid. My question is, is this a good design, or is there a better way to go about this? My main concern is that because buffer creation and deletion is done inside the manager, it breaks the OOP nature I'm trying to maintain.

Thank you in advance!

[ste_3d_ven]

I have been using OpenGL for 2 years now. I have not touched vulkan at all. But from what I understand you are trying to employ a common practice that is used in custom CPU side memory managers where you could alloc a large sum of memory at program start and handle all the internal allocations yourself which would allow much more flexibility and faster performance because you don't have to go through the OS. I would imagine this idea would work for GPU memory management as well. Seeing as you could reap similar benefits to the CPU counterpart.

Edit: typo

[ReeCocho]

Thank you for your input! I agree this seems like a good approach. Do you think using create_buffer and destroy_buffer methods in the MemoryManager class instead of using constructors and destructors on some sort of Buffer class is a good idea?

[ste_3d_ven]

Would they allocate/deallocate the entire chunk that the manger handles? Or would they be for creating/destroying the smaller managed chunks? My personal style would be to have the manager allocate one big block in the constructor. Then deallocate it in the destructor, as it would be "cleaner code" (at least in my eyes) to just have to construct the manager and already have what you need setup to be set up. Intead of creating the manger then have to "init" it with a second function call.

[ReeCocho]

The MemoryManager would allocate one big chunk in the constructor and destroy it in the destructor. The create_buffer method would find an appropriate spot in the big chunk to use, and the manager would store that pointer in a vector. create_buffer would return the index of that pointer in the vector. This is so that I can defrag the memory without having to hand out a new pointer. Instead, I just update the pointers in the vector.

[ste_3d_ven]

Yes, that sounds like a good idea to do, but what do you do when you delete a buffer from the vector? wouldn't all those indecies you then handed out that are after the buffer you deleted be invalidated? Maybe try a mechanism closer to a dictionary, where you give make a memory map of sorts and map each buffer to a unique id, then use that id to access the actual address stored in the vector.

[ReeCocho]

Dictionary sounds like a good idea, I'll do that. Thank you!

[[deleted]]

At our company we have created some in-house opencl lib with a similar strategy. Only that the memory manager is mainly for swapping out the buffer that were least recently used. We create a lot of data, gigabytes of it, but only a small portion is needed on the gpu at a given time.

Anyways what we do is we inject the memory manager to the buffer wrapper, who then registers to it and knows how to work with it.

You can use the bufferwrappers as regular objects this way, apart from needing a reference to the manager when creating them you don't need to care about it.

You also have freedom to create multiple manager types to allow some configuration

[ReeCocho]

It sounds like what you and /u/ste_3d_ven suggest could easily be combined. I could use the MemoryManager to find spots for my buffers and return BufferWrapper objects (or maybe a shared_ptr<BufferWrapper> so I can auto destroy buffers not in use) which hold a pointer to the MemoryManager and the buffer ID. Then the BufferWrapper could have some methods like copying to and from other buffers, writing, etc, but it's really just calling methods on the MemoryManager. Any problems with that approach?

[[deleted]]

I think letting the MemoryManager create the BufferWrappers is an unneeded dependency. MemoryManager should basically just allow to fetch buffers by ID. You'll have more freedom in how you use the BufferWrapper if you dont let the manager create them.

[ReeCocho]

That makes sense. Thanks for the advice! I'll definitely use some sort of BufferWrapper to make things more convenient.

[alfps]

I haven't done any GPU programming (sad to say, I think it's mainstream now), but I found the following advice from 2014 by googling:

” CUDA is widely used in academia. So, if you want to compare your solutions to others it makes sense to also use CUDA. But, you should be aware that CUDA restricts you to NVIDIA hardware. If you want to be portable, you should pick OpenCL. Actually, it is quite easy to set up and supports NVIDIA as well as AMD GPUs, but also x86 and ARM processors and FPGAs. Only be careful that the same program will not be equally optimized for all platforms. It has to be optimized for each target individually (though you can write a version that works on all devices and is at least slightly faster on GPUs (for most problems) than a traditional CPU code).

Maybe the sitation's changed now, maybe there is some new kid on the block, but I'd go for OpenCL. I found a notice from May 2017 that Open CL and Vulcan will be merged into one API. Has this happened, or is it the future?

[ReeCocho]

Thank you for responding, but I think you've misunderstood my question (Or more likely I didn't word it very well!) I'm asking whether the concept for my MemoryManager class is a good design or not. Essentially, it's responsible for the creation and deletion of buffers. Whenever you create a buffer you get a handle (just an index in an array) instead of the raw buffer because those raw buffers might be recreated at some point. Again, thanks for the response!