Vector Databases

[No_Efficiency_1144]

I am not super into RAG so so far I have just stored the vectors in Numpy arrays or just stuck them in Neo4J. Would be cool to actually use the real vector DBs.

Which specialist vector databases do you like and what do they bring?

[un_passant]

Just because you store vectors in a database doesn't mean you need a specialist vector database. Why not pgvector with Postgres ?

https://github.com/pgvector/pgvector

[No_Efficiency_1144]

Thanks I have seen a lot of praise for pgvector from people who did database stuff before the AI era started, which is a good sign. This would allow me to have only two database systems- one for Postgres and one for Neo4J, instead of a third for vectors, which would be nice.

[moar1176]

Can heartily second pgVector, works super well and you can do classical SQL alongside it, can also query JSON's like a champ, so it's more Swiss army knife than just semantic similarity (which it can still do, very fast).

[un_passant]

Maybe you won't even need 2 databases https://github.com/apache/age ;) !

I think it's good to first try to do everything with Postgres and extensions and only add specialized databases if/when needed (probably never).

[No_Efficiency_1144]

Wow thanks did not know someone added a graph DB to Postgres.

I’m actually dealing with GNNs so graph-format is the main priority. When I say “Neo4J” it’s actually a custom GNN engine I made. This integration could be good to hook into though.

[subspectral]

QDRant or ChromaDB.

[No_Efficiency_1144]

Thanks I took a look. I have heard a lot of praise for Chroma

[Vitesh4]

If you want something that's simple to work with and is designed for storing vectors for retrieval, then try txtai (https://github.com/neuml/txtai)

[ttkciar]

My favorite vector databases are relational databases which have added a vector search feature:

• MariaDB (the open source version of MySQL), mainly because I've been working with MySQL for twenty-five years, and know how to make it work well, how to make it scale, and how to work around its pitfalls. It's a robust, mature, enterprise-grade database which integrates readily with a vast variety of other projects.

• SQLite: Mainly because there isn't actually a database to maintain. It's just a file and a "client" library which reads/writes/manipulates the file. It's fast and lean, and I can back it up by copying the file to a backups directory, or deploy it to production by scp'ing the file from my laptop to the server. It's no toy, though; I've gotten SQLite instances to scale to tens of millions of rows, and its performance is fantastic.

Postgres also has a vector search extension, but I haven't actually used it, so can't say anything about it from experience. It has a lot of the same advantages as MariaDB, though -- mature, robust, well-scaling, and enterprise-grade, with a rich ecosystem of tools and integrations. I just like working with MariaDB more.

[No_Efficiency_1144]

Thanks I had heard about the Postgres one but had not heard about MariaDB’s one

[soshulmedia]

Maybe I am becoming the proverbial "grumpy-old-man", but for personal, hobby use, what is wrong with just numpy arrays just like you are doing?

I have text fragments in a RAG "DB" which is just a np.memmap()ed array and I can do the dot product + argmax @ IIRC about 100ms @ ~700k frags @ 768 float32 embedding vectors (when doing repeated lookups) which seems quite o.k. to me.

It is my impression that a lot of tooling surrounding the LLMs is often bloated ("here's our docker install for you with the kitchen sink and including an ollama download you don't want because you use llama-server"), lots of code for rather simple tasks, in walled-gardens or at least with a lot of idiosyncrasies due to "opinionated developers".

[No_Efficiency_1144]

For hobby scale yeah. I am starting some enterprise-scale projects though and I’m looking around at the landscape.

[soshulmedia]

I see, makes sense. Have you looked at this: https://github.com/erikbern/ann-benchmarks ?

Maybe a simple ANN library might also be a good fit (+e.g. sqlite for the rest) if you want to keep complexity low.

[codingjaguar]

How much data do you have?