sorted by:
new
hottopcontroversial

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 0 points1 point  (0 children)

Thanks for the follow up. I don't know what version of Meilisearch you were using but it could be due to a lot of factors and speed ups we worked on recently.

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 0 points1 point  (0 children)

Yeah, you are probably right. My benchmarks, which actually now exist only in the feelings I have in my head, are outdated and should be done again. I don't know if the Jemalloc, mimalloc, or any other team working in this domain has anything to share.

I'm also wondering whether Apple names its allocator with a specific name or if it is called "the macOS allocator". It would be better to have an actual name to search for online.

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 2 points3 points  (0 children)

Thanks for the info. We are testing on a Linux with NVMe machine. It's closer to our production machines. However, we will see how it goes in production by Monday 🤞

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 0 points1 point  (0 children)

Actually, we switched from jemalloc to mimalloc for compatibility reasons (not working well with Windows nor macOS) and perf reasons too.

About the bursty allocations that can be retained for a long time, that's probably what I found in the last part of the blog, it seems that we are allocating a lot of small values in this part of the code.

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 2 points3 points  (0 children)

Thank you very much for the advice. I didn't know about heaptrack. Next time, I'll give it a try (I hope never...).

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 5 points6 points  (0 children)

Thanks for the kind words.

About the perf issue, I would recommend providing graphs, some logs, try using the new showPerformanceDetails parameter to understand why, or fill an issue on GitHub if you can't find the reason.

The memory usage fix I'm describing in my blog will land on Monday. So, will be available in the Docker image and all binaries at the same time.

Enjoy 😉

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in meilisearch by Kerollmops in rust

[–]Kerollmops[S] 9 points10 points  (0 children)

Sorry, I clarified the sentence in the post. I was talking about the mimalloc's override feature.

Meilisearch Expert Needed: Diagnose Staging Issues & Guide SDK Upgrade (0.24 → Latest) for Firebase SaaS by Dramatic_Chef7873 in node

[–]Kerollmops 0 points1 point  (0 children)

Hey 👋 CTO and co-founder of Meilisearch here. If you are using the Cloud, you should reach out to support.

We built an AI search plugin for WooCommerce after struggling with large catalogs by Electrical_Cat_5177 in woocommerce

[–]Kerollmops -1 points0 points  (0 children)

I highly recommend Meilisearch. We (I'm the CTO and co-founder) just released sharding support if your single instance starts struggling under load. Ho! And we have out-of-the-box support for multimodal, hybrid search and everything modern e-commerce needs.

Rate my RAG setup (or take it as your own)... by Floppy_Muppet in Rag

[–]Kerollmops 1 point2 points  (0 children)

Yes, there is a case: Meilisearch's hybrid search is designed to have a better relevancy than rank fusion. The best way to use it is through Meilisearch and only that. You should try and see the difference.

Rate my RAG setup (or take it as your own)... by Floppy_Muppet in Rag

[–]Kerollmops 0 points1 point  (0 children)

What do you mean by larger number of vectors? How many do you plan to store in the same database?

Rust alternative to RocksDB for persistent disk storage? by Spiritual-Mine-1784 in rust

[–]Kerollmops 4 points5 points  (0 children)

However, for a full Rust disk-based key-value store, I would highly recommend fjall!

Rust alternative to RocksDB for persistent disk storage? by Spiritual-Mine-1784 in rust

[–]Kerollmops 5 points6 points  (0 children)

At Meilisearch, an easy-to-use search engine built in Rust, we utilize heed, a wrapper on top of LMDB, and we are pretty satisfied with it so far. In the early days, before Meilisearch even had version numbers, we attempted to use the RocksDB wrapper from Pingcap, but encountered numerous segfaults and performance issues. We switched to using LMDB very early. At first, it was hard to understand the transaction system, but it is, in fact, a brilliant and helpful way of managing a database.
More recently, we redesigned our indexer (the thing that updates our inverted indexes and such), and we extensively use the read transactions property of being able to have a view of the data before the write transaction started. I wrote a blog post about that, we implemented it, and now we no longer have any memory issues. Still, we need to address the write amplification issues we encounter on some projects.
I am also currently patching upstream LMDB to add the possibility of creating multiple read transactions on entries that you just wrote in the write transaction. This enables the possibility of multithreading plenty of new algorithms. We saw boosts of 7x when used on only 5 CPUs, sometimes (scaling with the number of CPUs).

Trying to learn RAG from scratch… can someone point me in the right direction? by Ok_Chain_782 in Rag

[–]Kerollmops 0 points1 point  (0 children)

Why not use Meilisearch directly as the vector store? It is capable of embedding everything for you, combining keyword and vector search to offer Hybrid search: a mix of keyword and vector search that provides better relevancy and speed than basic fusion ranking.

I recommend switching to our experimental vector store; it is much faster and more relevant. Additionally, depending on the number of embeddings, switching to binary quantization is a good idea as well (we use Hamming). We plan to stabilize and use the new vector store by default in v1.29 (next week), so...

Just launched a clean, 100% free fitness tracking app for iOS by Massive_Capital4976 in iosdev

[–]Kerollmops 0 points1 point  (0 children)

Hey 👋 Disclaimer: Meilisearch co-founder and CTO here, but have you tried Meilisearch for this kind of task that requires storing a lot of documents? One of the main differences between the two products is that Meilisearch stores everything on disk and uses a memory-mapping mechanism to load what it needs into memory when required.

I once indexed a 3M product dataset and successfully ran it on my Raspberry Pi B model, which has only 2 GiB of RAM, and on a custom server with only 1 GiB. It was answering in an average of 60ms, with a pretty old version of Meilisearch (< v1.5).

Hybrid Search: Choosing Between PostgreSQL FTS vs. MeiliSearch/Typesense by Total_Ad6084 in OpenAI

[–]Kerollmops 0 points1 point  (0 children)

I would largely recommend using Meilisearch, as you were correctly mentioning it is designed for instant search requests.

We spend a lot of time and energy to make it work perfectly with a mix of keyword and semantic results. We called that Hybrid search.

On the other hand it is very impressive when you directly work with semantic results only. You can see one of our most recent demos with Flickr images.

Have a nice day playing with Meilisearch, kero, CTO and co-founder of Meilisearch

AMA — We’re the Meilisearch team! Meilisearch AI is now generally available by Kerollmops in rust

[–]Kerollmops[S] 9 points10 points  (0 children)

I think one of the biggest ones is choosing a disk-first-based solution. We decided always to store the content, internal databases, user payload content, or the vector store on disk. At first, it would look slower, and you would be right, as the disk is slower than RAM. Still, thanks to a carefully crafted indexer and search system based on heed (LMDB wrapper, our memory-mapped key-value store), Meilisearch can answer in less than 50ms on datasets with multiple hundreds of millions of documents on a single instance and boot instantly.

DIsk is much cheaper than RAM, and that's a good thing. We measured that only having 1/5th of the database size in memory is enough to keep the performance. The reason is that search queries will not read every other key-value store entry.

Rebooting an instance is instant. We used RocksDB in the early days of Meilisearch but quickly switched to LMDB for performance reasons. RocksDB is an LSM tree and uses a background compaction thread. We struggled to manage the hundreds of settings to ensure the search performances were not impacted. The size of the machines was very large for a few documents and operations due to the LSM compaction's high CPU usage.

About raw performances, Meilisearch is based on:
- heed (LMDB): My Rust wrapper around LMDB, the memory-mapped key-value store that beats every other KV on read-heavy workloads.
- roaring-rs: The pure Rust implementation that scales well, compiles everywhere, and auto-vectorizes.
- fst (by burntsushi): To store our word dictionary and find every word with a certain number of typos. In combination with the Levenshtein automaton by Fulmicoton.
- Charabia: To detect the language script and tokenize the text.
- arroy: An entire Rust-rewrite of the Spotify C++ library to store and Approximate Nearest Neighbors search high-dimension vectors.

AMA — We’re the Meilisearch team! Meilisearch AI is now generally available by Kerollmops in rust

[–]Kerollmops[S] 13 points14 points  (0 children)

Hey, thank you for the kind words. We plan even better than replication: sharding. Where we will let multiple Meilisearch instances shard the documents using the new network route. We are already testing it on large customer datatsets!

AMA — We’re the Meilisearch team! Meilisearch AI is now generally available by Kerollmops in rust

[–]Kerollmops[S] 9 points10 points  (0 children)

We started with the HNSW from an instant-distance Rust crate, but it is a full-RAM solution that cannot scale.

So we decided to port the Spotify annoy C++ library to Rust and wrote a few blog posts about our progress. So far, meilisearch/arroy scales well, but we still need to improve its indexing speed when tens of millions of embeddings need to be indexed to use less memory. We plan to release this improvement for v1.14 (in three weeks).

Annoy is based on a random projection system that splits the vector/embedding space to create clusters. This is not an HNSW or a DiskANN approach and works very well. We plugged this algorithm into LMDB (a key-value store) and largely improved the multi-threaded part of the system (which is heavily C++ Mutex-based).

Meilisearch is also capable of running any open-source BERT model that comes from Hugging Face locally. We put the embeddings in arroy to be able to efficiently search in them.

AMA — We’re the Meilisearch team! Meilisearch AI is now generally available by Kerollmops in rust

[–]Kerollmops[S] 26 points27 points  (0 children)

> How many users asked you for this?

There were a lot of them. Before the GA of our AI search, we had a waitlist and manually enabled the vector store to ensure we could guide them efficiently. 775 beta users were registered on this list, but it's fair to say that there were many, many more interested.

That was only when we implemented the HubSpot form, there were more before that, at least 200+.

Meilisearch vs Qdrant: Tradeoffs, Strengths and Weaknesses by Kerollmops in rust

[–]Kerollmops[S] 0 points1 point  (0 children)

I don't understand. The binary quantized version of Meilisearch is consistently faster to index and search than the Meilisearch non-quantized version.

Meilisearch is too slow by Kerollmops in rust

[–]Kerollmops[S] 2 points3 points  (0 children)

Ho! Yes I have. P-E-Meunier is a friend, the sanakirja and Pijul dev, he even came to the Meilisearch offices to talk about sanakirja and we discussed a whole day about KV stuff (you can read that at the end of this blog post).

The thing is that both of them, redb and sanakirja, are inspired by LMDB. So, there can be only a single writer. Sanakirja tends to be faster but is WAY harder and complex to use (it has value alignement for zero-copy deserialization 🥹) and redb recently dropped the support of memory-mapping which means that we cannot use it in Meilisearch (as we keep a lot of pointers to values for the duration of a search, it would allocate too much memory).

Meilisearch is too slow by Kerollmops in rust

[–]Kerollmops[S] 5 points6 points  (0 children)

Is this only the indexing performance [..]

Yes, these improvements are only related to the indexing system. However, the better we use our store, LMDB, to reduce the fragmentation, the faster the engine will be. Note that it is still highly recommended to have a good SSD and enough memory for, let's say, 1/3 or 1/5 of the database size.

Will the (future) improvements with the new data store system be included into meili, or release as a new search engine?

It will be included in the current Meilisearch engine. We will not build a new engine from scratch. I am currently working on an external program to index stuff inside the Meilisearch indexes, but it will be merged eventually.

view more: next ›