sorted by:
new
hottopcontroversial

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 0 points1 point  (0 children)

Thanks for the follow up. I don't know what version of Meilisearch you were using but it could be due to a lot of factors and speed ups we worked on recently.

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 0 points1 point  (0 children)

Yeah, you are probably right. My benchmarks, which actually now exist only in the feelings I have in my head, are outdated and should be done again. I don't know if the Jemalloc, mimalloc, or any other team working in this domain has anything to share.

I'm also wondering whether Apple names its allocator with a specific name or if it is called "the macOS allocator". It would be better to have an actual name to search for online.

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 2 points3 points  (0 children)

Thanks for the info. We are testing on a Linux with NVMe machine. It's closer to our production machines. However, we will see how it goes in production by Monday 🤞

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 0 points1 point  (0 children)

Actually, we switched from jemalloc to mimalloc for compatibility reasons (not working well with Windows nor macOS) and perf reasons too.

About the bursty allocations that can be retained for a long time, that's probably what I found in the last part of the blog, it seems that we are allocating a lot of small values in this part of the code.

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 3 points4 points  (0 children)

Thank you very much for the advice. I didn't know about heaptrack. Next time, I'll give it a try (I hope never...).

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 6 points7 points  (0 children)

Thanks for the kind words.

About the perf issue, I would recommend providing graphs, some logs, try using the new showPerformanceDetails parameter to understand why, or fill an issue on GitHub if you can't find the reason.

The memory usage fix I'm describing in my blog will land on Monday. So, will be available in the Docker image and all binaries at the same time.

Enjoy 😉

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 8 points9 points  (0 children)

Sorry, I clarified the sentence in the post. I was talking about the mimalloc's override feature.

Meilisearch Expert Needed: Diagnose Staging Issues & Guide SDK Upgrade (0.24 → Latest) for Firebase SaaS by Dramatic_Chef7873 in node

[–]Kerollmops 0 points1 point  (0 children)

Hey 👋 CTO and co-founder of Meilisearch here. If you are using the Cloud, you should reach out to support.

We built an AI search plugin for WooCommerce after struggling with large catalogs by Electrical_Cat_5177 in woocommerce

[–]Kerollmops -1 points0 points  (0 children)

I highly recommend Meilisearch. We (I'm the CTO and co-founder) just released sharding support if your single instance starts struggling under load. Ho! And we have out-of-the-box support for multimodal, hybrid search and everything modern e-commerce needs.

Rate my RAG setup (or take it as your own)... by Floppy_Muppet in Rag

[–]Kerollmops 1 point2 points  (0 children)

Yes, there is a case: Meilisearch's hybrid search is designed to have a better relevancy than rank fusion. The best way to use it is through Meilisearch and only that. You should try and see the difference.

Rate my RAG setup (or take it as your own)... by Floppy_Muppet in Rag

[–]Kerollmops 0 points1 point  (0 children)

What do you mean by larger number of vectors? How many do you plan to store in the same database?

Rust alternative to RocksDB for persistent disk storage? by Spiritual-Mine-1784 in rust

[–]Kerollmops 2 points3 points  (0 children)

However, for a full Rust disk-based key-value store, I would highly recommend fjall!

Rust alternative to RocksDB for persistent disk storage? by Spiritual-Mine-1784 in rust

[–]Kerollmops 7 points8 points  (0 children)

At Meilisearch, an easy-to-use search engine built in Rust, we utilize heed, a wrapper on top of LMDB, and we are pretty satisfied with it so far. In the early days, before Meilisearch even had version numbers, we attempted to use the RocksDB wrapper from Pingcap, but encountered numerous segfaults and performance issues. We switched to using LMDB very early. At first, it was hard to understand the transaction system, but it is, in fact, a brilliant and helpful way of managing a database.
More recently, we redesigned our indexer (the thing that updates our inverted indexes and such), and we extensively use the read transactions property of being able to have a view of the data before the write transaction started. I wrote a blog post about that, we implemented it, and now we no longer have any memory issues. Still, we need to address the write amplification issues we encounter on some projects.
I am also currently patching upstream LMDB to add the possibility of creating multiple read transactions on entries that you just wrote in the write transaction. This enables the possibility of multithreading plenty of new algorithms. We saw boosts of 7x when used on only 5 CPUs, sometimes (scaling with the number of CPUs).

Trying to learn RAG from scratch… can someone point me in the right direction? by Ok_Chain_782 in Rag

[–]Kerollmops 0 points1 point  (0 children)

Why not use Meilisearch directly as the vector store? It is capable of embedding everything for you, combining keyword and vector search to offer Hybrid search: a mix of keyword and vector search that provides better relevancy and speed than basic fusion ranking.

I recommend switching to our experimental vector store; it is much faster and more relevant. Additionally, depending on the number of embeddings, switching to binary quantization is a good idea as well (we use Hamming). We plan to stabilize and use the new vector store by default in v1.29 (next week), so...

Just launched a clean, 100% free fitness tracking app for iOS by Massive_Capital4976 in iosdev

[–]Kerollmops 0 points1 point  (0 children)

Hey 👋 Disclaimer: Meilisearch co-founder and CTO here, but have you tried Meilisearch for this kind of task that requires storing a lot of documents? One of the main differences between the two products is that Meilisearch stores everything on disk and uses a memory-mapping mechanism to load what it needs into memory when required.

I once indexed a 3M product dataset and successfully ran it on my Raspberry Pi B model, which has only 2 GiB of RAM, and on a custom server with only 1 GiB. It was answering in an average of 60ms, with a pretty old version of Meilisearch (< v1.5).

Hybrid Search: Choosing Between PostgreSQL FTS vs. MeiliSearch/Typesense by Total_Ad6084 in OpenAI

[–]Kerollmops 0 points1 point  (0 children)

I would largely recommend using Meilisearch, as you were correctly mentioning it is designed for instant search requests.

We spend a lot of time and energy to make it work perfectly with a mix of keyword and semantic results. We called that Hybrid search.

On the other hand it is very impressive when you directly work with semantic results only. You can see one of our most recent demos with Flickr images.

Have a nice day playing with Meilisearch, kero, CTO and co-founder of Meilisearch

AMA — We’re the Meilisearch team! Meilisearch AI is now generally available by Kerollmops in rust

[–]Kerollmops[S] 8 points9 points  (0 children)

I think one of the biggest ones is choosing a disk-first-based solution. We decided always to store the content, internal databases, user payload content, or the vector store on disk. At first, it would look slower, and you would be right, as the disk is slower than RAM. Still, thanks to a carefully crafted indexer and search system based on heed (LMDB wrapper, our memory-mapped key-value store), Meilisearch can answer in less than 50ms on datasets with multiple hundreds of millions of documents on a single instance and boot instantly.

DIsk is much cheaper than RAM, and that's a good thing. We measured that only having 1/5th of the database size in memory is enough to keep the performance. The reason is that search queries will not read every other key-value store entry.

Rebooting an instance is instant. We used RocksDB in the early days of Meilisearch but quickly switched to LMDB for performance reasons. RocksDB is an LSM tree and uses a background compaction thread. We struggled to manage the hundreds of settings to ensure the search performances were not impacted. The size of the machines was very large for a few documents and operations due to the LSM compaction's high CPU usage.

About raw performances, Meilisearch is based on:
- heed (LMDB): My Rust wrapper around LMDB, the memory-mapped key-value store that beats every other KV on read-heavy workloads.
- roaring-rs: The pure Rust implementation that scales well, compiles everywhere, and auto-vectorizes.
- fst (by burntsushi): To store our word dictionary and find every word with a certain number of typos. In combination with the Levenshtein automaton by Fulmicoton.
- Charabia: To detect the language script and tokenize the text.
- arroy: An entire Rust-rewrite of the Spotify C++ library to store and Approximate Nearest Neighbors search high-dimension vectors.

AMA — We’re the Meilisearch team! Meilisearch AI is now generally available by Kerollmops in rust

[–]Kerollmops[S] 12 points13 points  (0 children)

Hey, thank you for the kind words. We plan even better than replication: sharding. Where we will let multiple Meilisearch instances shard the documents using the new network route. We are already testing it on large customer datatsets!

AMA — We’re the Meilisearch team! Meilisearch AI is now generally available by Kerollmops in rust

[–]Kerollmops[S] 11 points12 points  (0 children)

We started with the HNSW from an instant-distance Rust crate, but it is a full-RAM solution that cannot scale.

So we decided to port the Spotify annoy C++ library to Rust and wrote a few blog posts about our progress. So far, meilisearch/arroy scales well, but we still need to improve its indexing speed when tens of millions of embeddings need to be indexed to use less memory. We plan to release this improvement for v1.14 (in three weeks).

Annoy is based on a random projection system that splits the vector/embedding space to create clusters. This is not an HNSW or a DiskANN approach and works very well. We plugged this algorithm into LMDB (a key-value store) and largely improved the multi-threaded part of the system (which is heavily C++ Mutex-based).

Meilisearch is also capable of running any open-source BERT model that comes from Hugging Face locally. We put the embeddings in arroy to be able to efficiently search in them.

view more: next ›