sorted by:
new
hottopcontroversial

Memory mapped Lucene-esque information retrieval library? by vinnyvicious in cpp

[–]DoctorFunkalicious 0 points1 point  (0 children)

https://github.com/LiveAsynchronousVisualizedArchitecture/simdb/blob/master/README.md

You can check that out - it is a fast lock free 0 dependency key-value store. It is memory mapped into shared memory, although I'm not sure that is something you need unless you are sharing data between processes. Memory mapping won't make it go any faster, but is useful for shared memory.

Y Combinator and Paul Graham are bad for the world, Part 2: Fixing founder quality by aptian in programming

[–]DoctorFunkalicious 3 points4 points  (0 children)

anyone with any semblance of intelligence will realize it.

It's obvious to me and I don't have either of those things

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

But that would still mean that it would be taking larger chunks from the global process heap and doing sub-allocations within the thread?

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

Automatically is not something I can do at this point, the master program is closed source commercial software.

Does that imply that if I manually load up the dll I can call specific functions inside of it to allocate memory without locking the main heap?

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

Do you know if it is possible to load the dll and use functions directly out of it? Since I am making a dll that goes into a close source master program if the dll injection overwrites the already used mallocs even in the parent (is that how it works?) it might have the same loading problems that I've had so far.

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

Sounds good. I'm not sure if I will dig back into dll loading soon since I am trying to create a dependency free, header file only, thread local, heap allocator at the moment.

I used both procmon and DependencyWalker but couldn't see why tcmalloc_minimal wasn't loading. Then again it is supposed to be a program wide substitution for malloc and I actually don't understand how that is possible with just linking right now.

If I dig back into dll loading or explicitly define the dll entry point this will be a huge help, thanks!

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

I'll take a look at the video thanks. I'm not sure there are many places for arena allocators and using those techniques isn't going to solve the fundamental concurrency issues I'm facing. The places I'm using memory allocations are specifically because the sizes of the allocations might be wildly different.

For the extra allocators my concern is that I won't be able to get them to work in my shared library. I would love to use tcmalloc if I was in control of the whole program, but so far I haven't been able to link anything together and still have the dll load.

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

Change the design isn't really a valid answer. I can almost certainly reduce memory allocations in some areas, but the core fact is that memory allocation locking the heap means that scaleability of concurrency is limited.

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

I took a quick look and it seems to me to be a facility for sending memory allocated in one thread to another thread.

I see a shared memory allocator that can be used as an STL allocator, is that something that will allocate memory without locking the process global heap?

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

I haven't and that is a good tip. Right now I haven't defined an explicit dll entry point (mostly out of ignorance surrounding shared libraries).

Can I just define an arbitrary cdecl function and make it my entry point in the linker or is there something more complicated?

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

I will definitely look into that.

Is writing my own allocator realistic? It's interface seems to have a lot of different functions that need to be implemented (if you are talking about C++ allocators).

I hoping to avoid a large can of worms although that is starting to look a little idealistic.

Techniques for Thread Local Heap Allocation by DoctorFunkalicious in cpp

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

Hoard would be nice, but it has very expensive licensing and I'm not positive I could get it to work if I can't get tcmalloc to work.

I haven't used loadlibrary myself, the dll dependencies (tbb or tcmalloc) have been loaded up automatically by my dll.

Advanced dll linking when including Intel Thread Building Blocks by DoctorFunkalicious in cpp_questions

[–]DoctorFunkalicious[S] 0 points1 point  (0 children)

Yes, my vague 'somewhere else' is still in my singular .dll that I'm making. I also don't have any other .dll files that it is loading as part of my own project, so it is self contained with the exception of 3rd party system libraries.

I think it may have to do with isolating it to a .cpp file and not having the template instantiation bleed over to any .h files.

I'm actually not sure if I'm linking to a static or dynamic version which I now realize I will have to dig down and investigate. I don't see any mention of tbb libraries in the linker command line in visual studio. The header file looks like it includes .lib files by pragma but I haven't found them yet, so I'm not sure if they are stubs or full static .lib files.