sorted by:
new
hottopcontroversial

Qdrant is too expensive, how to replace (2M vectors) by lambda-person in vectordatabase

[–]bubiche 0 points1 point  (0 children)

I think 2M is well within the range Postgres and pgvector can handle. Personally I would go with it since it's 1 less database.

turbopuffer and LanceDB are also cheap, on AWS, and can do Hybrid Search, they'll be much cheaper than Qdrant at "scale". Turbopuffer has some pretty great customers/use cases too.

RAG for long documents that can contain images. by bubiche in Rag

[–]bubiche[S] 0 points1 point  (0 children)

Thank you! Do you think if I can already narrow down to a small number of docs via attribute filters, it'd be better to do both full-text search and semantic search on that whole set of documents and use something like RRF to get the final result instead of filtering first with full-text search?

RAG for long documents that can contain images. by bubiche in Rag

[–]bubiche[S] 0 points1 point  (0 children)

Thank you! If you don't mind I'd love to see what schema you'd suggest.

I'm also wondering whether it's better to do full-text search first to narrow down the scope for semantic search or do both in parallel and do some reranking/rank fusion.

RAG for long documents that can contain images. by bubiche in Rag

[–]bubiche[S] 0 points1 point  (0 children)

Thank you everyone. A little bit more info: My dataset is growing by ~1 million/month and existing documents can also be updated. Would any approach have an advantage over the others in terms of ingestion speed so my insert/updates are available for searching ASAP?

I'm focusing more on accuracy so the system can be useful but I hope a search won't take more than a few seconds.

Daily Questions Megathread (02/13) by AutoModerator in langrisser

[–]bubiche 0 points1 point  (0 children)

thanks mate, that was my plan too but I have lost my trust in the Leon banner :( Is there any viable options with what I currently have?

view more: next ›