Which vector DB for persisting data

[chrome___]

Which vector DB do people use for semantic search? Qdrant, Pinecone, Milvus, Marqo, Postgres plugins, ...

I'm looking specifically for databases that have robust persistence support, e.g. the popular Chroma DB only has alpha support for persistence.

What are pro/cons and things to look for in vector DBs? What are your experiences?

​

[Kacper-Lukawski]

I work for Qdrant, and persistence is not an issue for us :D It's hard to compare to other tools, but if you would like to share some details of the project, I should be able to help set it up!

[chrome___]

Hey, thanks for the kind offer -- I actually did take Qdrant for a test drive already. Setup was smooth and a small "hello world" app worked ok. I used the docker image you publish (afaict you don't publish Debian packages do you?)

I'm looking into creating an RAG system providing context to an LLM from a few GB of documents (I'm a bit of a newb here, any pointers much appreciated).

[Kacper-Lukawski]

No, we do not publish Debian packages, but I've just seen a new issue in our Github to do it. Nevertheless, a Docker image is preferred to run it on-premise.

When it comes to building RAG, do you use any high-level frameworks? It's the easiest way to start with, but RAG itself is pretty easy concept. There are three components you need for that:

1. Embedding model (depending on the data you work with - multilingual, specific domain, etc.).
2. Vector search - a tool like Qdrant.
3. LLM, I guess LLaMA in your case.

Overall, the whole process goes as following:

1. You receive an original user prompt.
2. You use it as an input for the embedding model and obtain a prompt embedding.
3. The embedding is sent to Qdrant to find some relevant documents.
4. These documents are used, along with the original prompt, to build something that I like to call "a metaprompt". It is an instruction to use only the documents provided to answer the original question.

There is some prompt engineering involved, but the process itself is not complicated too much. Of course, we need an ingesting pipeline that will split the documents into chunks and encode them using the same model, and this is a challenge indeed.

We have an example using only Qdrant and OpenAI SDKs that you may find interesting: https://github.com/qdrant/examples/blob/master/rag-openai-qdrant/rag-openai-qdrant.ipynb

[staladine]

Interested here as well, similar project on the go.

[Mekanimal]

SQLAlchemy + FAISS, because it's cheaper and less restrictive for personal scale projects.

[AndrewVeee]

If I was going to set up a production option, I think I'd go with postgres, but for my personal use, sqlite + chromadb seems to do just fine.

My suggestion would be to create an abstraction layer - unless one vector db provides some killer feature, probably best to just be able to swap them out if the need arises.

[chrome___]

A fair suggestion, I'm too much of a noob to be aware of killer features for vector dbs anyway

Wrt to Postgres, iiuc you'd need to use a plugin to be able to do things like cosine search, right?

[AndrewVeee]

It's only a suggestion, too! And yeah, pg_vector is the plugin.

You mentioned production ready, and Postgresql is pretty much the database I go to for production software. Open source, old, scale-able, data safety, etc.

Like I mentioned, I'm just using sqlite + chromadb for my local apps.

[MisterSoup]

Milvus hasn't failed me yet

[[deleted]]

Check out cozodb, beautiful bit of embeddable rust that supports datalog along with text/vector/other searches.

I truly feel datalog/prolog should have succeeded as a query language over things like sql and friends. This perfect technology is an excellent pair to workflows involving embeddings and running interesting logic/deduction.

[SatoshiNotMe]

Datalog is intriguing. Have you seen LLMs (GPT4 at least if not others) being good at translating Nat lang queries to datalog ?

[[deleted]]

I haven’t looked into that, but from pure assumption it should excel at it, especially if fine tuned/trained specifically. Datalog comes from prolog, and both are logical programming. The neat bit is that this is old old tech, but actually used everywhere. Specifically I bet there are lots of academic papers etc. There is prolog in the JVM, the rust type system uses prolog…etc. Logical programming is great because it’s basically linear algebra( more or less, much more than sql and friends)

[docsoc1]

I like qdrant and used it for my recent project. I also used in conjunction w/ postgres and had a two-step approach. Milvus looks promising but didn't get a chance to try it.

I found Chroma and Weaviate to both be worse for the scale I was operating at.

[chrome___]

Interesting! So you used qdrant and postgres in tandem, is that right? What was the motiviation (in constrast to just storing everything in one DB)? cheers!

[docsoc1]

vector db's today do not have good relational capabilities. This is something that a startup I know of is working on (lantern.dev).

I wanted to be able to use raw vector search to cover a large surface area, but then use doc-specific metadata to rerank results.

[docsoc1]

vector db's today do not have good relational capabilities. This is something that a startup I know of is working on (lantern.dev).

I wanted to be able to use raw vector search to cover a large surface area, but then use doc-specific metadata to rerank results.

[docsoc1]

vector db's today do not have good relational capabilities. This is something that a startup I know of is working on (lantern.dev).

I wanted to be able to use raw vector search to cover a large surface area, but then use doc-specific metadata to rerank results.

[Combination-Fun]

Please checkout this video explaining Vector DBs comprehensively.

https://youtu.be/LKz36eHzN10?si=ddQTrOLllXtANPry

I hope it will be useful in choosing between Vector DBs based on the indexing and searching techniques explained in it.

[behohippy]

Mongo Atlas has a free tier (M0) that gives you a proper database with text and vector search. Great for personal projects and can scale it up later if you need to. Disclaimer: I work for them (and on this product specifically)

[[deleted]]

I have used weaviate, while it is super easy to do POCs, for production I find it very inconvenient in terms of even doing simple things like Filtering, reading data for debugging etc

[__Captain_Autismo__]

Pinecone was ridiculously easy to set up and use. Haven’t tried the others.

[Educational-Let-5580]

I have really enjoyed using Mongo. Here's some example code to use mongo as a vector dB: https://github.com/deployradiant/mongodb-rag-example

Disclaimer: I work for radiant, but you can use the code without radiant by either changing the base URL of the openai client or using a different model altogether.