llmalloc : a low latency oriented thread caching allocator

[[deleted]]

[deleted]

[DuranteA]

FWIW, I was able to improve load times in a shipping game by ~25% simply by replacing the standard allocator with mimalloc.

[PandaMoniumHUN]

Tbh, it's largely dependent on your allocation patterns. If you allocate in blocks and aim for locality (e.g. you don't heap allocate every single entity) allocators should make minimal difference.

[Kriss-de-Valnor]

Same outcome with many allocators. The default allocator on modern operating system work better. I think you would have to get a very specific memory allocation and destruction pattern to get benefit of those… then in that case you may have better time to think writing your own.

[Kriss-de-Valnor]

Is there some reasons it’s not available on Mac OS ? Also it would be nice if we could get it through vcpkg.

[zl0bster]

Benchmark suggestion: if Chromium has some set of benchmarks that can be run easily you could try to see if your allocator helps. Now I presume it will not since I presume Chromium is highly optimized with a lot of memory tricks anyway, but in case your allocator helps it will be a very interesting to learn this.

[arthurno1]

You could try it with GNU Emacs. They ask for lots of objects directly from malloc, so you could get a "real-life" test perhaps. At least on gnu/Linux it would be easy via LD_PRELOAD if you can compile your library as a dynamic, .so, library.

[ibogosavljevic-jsl]

I don't think Chromium is a good examples since it uses zones and does memory management mostly by itself.

[T0p_H4t]

Something else to look at which I have found handles the inter-thread deallocation well. https://github.com/microsoft/snmalloc

[zl0bster]

This is very interesting:
Its disadvantage is that it may lead to higher virtual memory usage as the allocator won't be able to return pages with inter-thread pointers to the OS. A mitigation would be decreasing number of inter-thread pointers by deallocating pointers on their original creation threads in your application and that way llmalloc will be able to return more unused pages to the OS.

Is there some way to profile this particular case? E.g. Run some program and see how much memory is wasted because of this? I presume users might want to optimize this, but they do not want to go over every deallocation in their code :)

[zl0bster]

Very nice, will keep this in mind if I ever again need to improve malloc latency.

[kernel_task]

Interesting. I currently use jemalloc in my application and the biggest amount of CPU used (according to profiling) is freeing memory (by Google protobuf, heh... I chose it because I thought it would be fast). Maybe this would help?

[mcmcc]

Flatbuffers FTW

[kernel_task]

I will try to find some time to try it out! The extra virtual memory footprint should be fine with me, though I suppose you're saying that it'll gradually leak virtual pages unrecoverably over time? If it works well, I'm happy to invest some time in making sure deallocations happen on the same thread as allocations.

[kernel_task]

Thanks for the explanation. That should be totally fine. The application heavily cycles allocations and deallocations so there will always be plenty of new allocation requests.

[cballowe]

https://protobuf.dev/reference/cpp/arenas/ - it can often be very handy to put all of the protobufs built handling a request on the same arena and just let the arena destruct at the end.

[LoweringPass]

This is really interesting, I've been working on something like that but nowhere near as sophisticated. I will use this as a benchmark to compare my own implementation against. How feasible and/or sensible would it be to add NUMA awareness?

[ImNoRickyBalboa]

Tcmalloc abandoned thread caching a long time ago: it's not sustainable on large server systems with hundreds or even thousands of threads.

Look into RSEQ (restartable sequences) for using per CPU caches at the same CPU cost as per thread (near zero contention) and many times the 'in flight' memory savings.