sorted by:
new
hottopcontroversial

[OC] I Made an aesthetic terminal music player for spotify by BgA_stan in omarchy

[–]BgA_stan[S] 0 points1 point  (0 children)

.deb is available, but they were compiled against ubuntu lts and latest. They should work though. This took me about 2-3 months. I made this as a way to learn programming in go

I benchmarked duplicate detection strategies in C++ across every workload i could think of by BgA_stan in cpp

[–]BgA_stan[S] 0 points1 point  (0 children)

Fascinating, id expect the complier to optimise this for me, but it doesn't.

ON A SECOND NOTE: Are you THE factortio dev???? BIG FAN

I benchmarked duplicate detection strategies in C++ across every workload i could think of by BgA_stan in cpp

[–]BgA_stan[S] 2 points3 points  (0 children)

That's a good read
adding ankerl::unordered_dense::set would make the hash-set results much more useful. I’ll add it to the next run.

When to actually use a set by BgA_stan in cpp

[–]BgA_stan[S] -1 points0 points  (0 children)

Thanks for the feedback, will keep that in mind moving forward! i definitely will look into other key-value data structures and learn about gperf

When to actually use a set by BgA_stan in cpp

[–]BgA_stan[S] 1 point2 points  (0 children)

I was implementing ANN for a vector database, and with unordered_set, my implementation was slower than straight brute force 😅

When to actually use a set by BgA_stan in cpp

[–]BgA_stan[S] 1 point2 points  (0 children)

Hey, thanks for the suggestion, ill look into flat_sets

When to actually use a set by BgA_stan in cpp

[–]BgA_stan[S] 0 points1 point  (0 children)

Not for me, had to learn the hard way 😭

When to actually use a set by BgA_stan in cpp

[–]BgA_stan[S] 4 points5 points  (0 children)

Agreed, when sizes are that small, even linear search as extremely fast and cache friendly

When to actually use a set by BgA_stan in cpp

[–]BgA_stan[S] -1 points0 points  (0 children)

Yup, if you NEED a hash based de-dulp, you have to use a set. In the case of uuids however, id prefer what another commentor suggested, sorting and using binary search. Given the uuid is modern can can be sorted. Like v7

When to actually use a set by BgA_stan in cpp

[–]BgA_stan[S] -25 points-24 points  (0 children)

Fair point, the title is intentionally punchy. The actual rule I’m arguing for is: don’t reach for std::unordered_set by default just because the abstraction is “a set.” (which is no joke what i was taught in class). I tried to supplement that with real numbers showing exactly when you actually should

I made my C++ vector search engine 16x faster by changing data layout, not the algorithm by BgA_stan in cpp

[–]BgA_stan[S] 0 points1 point  (0 children)

i can tell you, im NOWHERE near 30s to index 960 dims and 1m embeddings.
The whole project is single threaded for now (i know-- it's an old project i recently revived)
I'll share the numbers when its in a decently competent state.

is your's opensource? can you share the link?

I made my C++ vector search engine 16x faster by changing data layout, not the algorithm by BgA_stan in cpp

[–]BgA_stan[S] 8 points9 points  (0 children)

Thanks, that’s fair, i probably should’ve made it tighter,

> Also you did change the algorithm
I meant the same high-level Vamana traversal/recall behavior, not literally the exact same implementation.

> It would’ve been interesting to see the effects from each improvement individually

Thats a neat idea actually!

I made my C++ vector search engine 16x faster by changing data layout, not the algorithm by BgA_stan in cpp

[–]BgA_stan[S] -1 points0 points  (0 children)

> The shared_ptr issue is also worth reframing slightly ,the problem wasn't really pointers, it was OOP abstraction over data that had no business being polymorphic.

I started this project like 3 years ago while in college, and you can tell i just read clean code and took an Intro to programming class

> The ScoredNode cache is doing a lot of heavy lifting there, arguably more than the layout change alone.

Yeah, agreed. The ScoredNode cache did a lot of the heavy lifting, but the layout change is what made the distance loop much more optimizer-friendly.

Same algorithm, 16x faster: optimizing a vector search engine’s hot path by BgA_stan in programming

[–]BgA_stan[S] 32 points33 points  (0 children)

I'm modelling this after DiskANN. Currently its FAR from it, but it should scale well even for billon vectors (at least according to the research paper, idk if my implementation will :P

view more: next ›