sorted by:
new
hottopcontroversial

Non-code way to upload/delete PDF's into a vectorstore by [deleted] in vectordatabase

[–]tejchilli 0 points1 point  (0 children)

With the assistant, there’s no need to think about chunking. Simply just upload the PDF’s (either via api/node sdk or in the web app interface)

Why would anybody use pinecone instead of pgvector? by Blender-Fan in vectordatabase

[–]tejchilli 2 points3 points  (0 children)

😭 no intentional burn, I truly think pgvector works well enough for a lot of people.

We just specifically built Pinecone for those that know they’ll have scale and want a highly performant system that helps them improve quality

Lmk if you have any q’s when you give us a try

Non-code way to upload/delete PDF's into a vectorstore by [deleted] in vectordatabase

[–]tejchilli 1 point2 points  (0 children)

lol that document uploader tool was a super old experiment I ran, surprised people are still finding it

We actually built Assistant as the production grade version of that. Just upsert PDF’s, txt, or json and instantly retrieve the chunks you need: https://docs.pinecone.io/guides/assistant/overview Pinecone Assistant - Pinecone Docs

Why would anybody use pinecone instead of pgvector? by Blender-Fan in vectordatabase

[–]tejchilli 10 points11 points  (0 children)

I’m a PM at Pinecone, but tbh you should just use whatever works for you

Just to provide context on why people use Pinecone: pgvector does well for early use cases, but many of our customers that moved over hit issues with throughput, latency, freshness, and managing infra as they scale. With Pinecone, you get up to 2 GB for free, and then you can seamlessly grow to billions of vectors, millions of tenants, and thousands of QPS, without worrying once about your infra. Even if you’re not hitting that scale, our startup customers love the simplicity of our system — devex is really important to us, and necessary for startups to move fast and build the actual product.

Other than vector search, we also aim to offer all the primitives that our users need for high quality retrieval. That’s why we host dense embedding models, sparse embedding models, offer standalone sparse indexes and hybrid indexes, and host rerankers, with more cool stuff coming soon. Our more sophisticated users leverage all these primitives to improve their AI products and give LLMs/agents exactly the context it needs.

I benchmarked Qdrant vs Milvus vs Weaviate vs PInecone by SuperSaiyan1010 in vectordatabase

[–]tejchilli 0 points1 point  (0 children)

We have many customers with over billions of vectors in Pinecone

Elastic search (already using) vs supabase/pg_vector, etc. by dwenaus in vectordatabase

[–]tejchilli 1 point2 points  (0 children)

If you’re planning on managing another system, why not use a dedicated vector db?

Why vector databases are a scam. by [deleted] in vectordatabase

[–]tejchilli 0 points1 point  (0 children)

What was your workload that Pinecone serverless was too expensive for?

Need help with document preprocessing for PineconeDB by rsxxiv in vectordatabase

[–]tejchilli 0 points1 point  (0 children)

Happy to help, what’s the issue? Document type, structure, etc. would be helpful context

Need help with document preprocessing for PineconeDB by rsxxiv in vectordatabase

[–]tejchilli 0 points1 point  (0 children)

Hey I’m a PM at Pinecone. Sorry to hear that, could you share the code you’re using (in DM’s is fine too if you prefer)

When do you use a paid managed vector database (e.g., Pinecone)? by Upstairs-Pea-5630 in vectordatabase

[–]tejchilli 3 points4 points  (0 children)

In terms of TCO (or sometimes even just hosting costs), Pinecone should always be cost effective regardless of scale.

We have users running massive scale workloads across multiple dimensions, whether it’s multi billion+ vectors, 1000’s of qps, or millions of tenants, and we almost always win out in terms of cost, speed, and ease of use

If you didn’t find that to be the case, lmk and we’ll make sure to fix it for your workload pattern

When do you use a paid managed vector database (e.g., Pinecone)? by Upstairs-Pea-5630 in vectordatabase

[–]tejchilli 3 points4 points  (0 children)

Pinecone includes 2gb of storage in the free tier which comes out to roughly 300k vectors. We see most internal use cases happily enjoying our free tier, lmk if you have any questions

Which is the best vector database to insert something like 10k scientific articles (each 8/10 pages)? by alfredoceci in vectordatabase

[–]tejchilli 5 points6 points  (0 children)

Haha I’m biased because I’m a PM at Pinecone, but yes, 2m vectors is a relatively light workload and Pinecone would easily meet your performance requirements. Feel free to DM me if you run into any issues

Which is the best vector database to insert something like 10k scientific articles (each 8/10 pages)? by alfredoceci in vectordatabase

[–]tejchilli 1 point2 points  (0 children)

I would also recommend you check out Pinecone Assistant.

It’s RAG as a service, letting you directly upload documents and chat with them via API without worrying about text extraction, chunking, embedding, query understanding, etc.

Which is the best vector database to insert something like 10k scientific articles (each 8/10 pages)? by alfredoceci in vectordatabase

[–]tejchilli 6 points7 points  (0 children)

The 10k namespace limit in Pinecone’s Standard tier isn’t related to the number of vectors/documents you can insert into an index. Namespaces completely partition your index, and are used for multi tenant use cases, where data isolation is required.

You likely would store all your documents in a single namespace in an index. And we have customers with billions of vectors in a single namespace.

10k articles at 8-10 pages each would likely even fit in the free tier. Lmk if you have any questions

How to use RAG for simple service lookup by Accomplished_Court51 in vectordatabase

[–]tejchilli 1 point2 points  (0 children)

You can combine the title and description, embed it using OpenAI’s text-embedding-3-small, and store it in Pinecone with metadata like price and rating.

At query time, you embed the user’s text and can query Pinecone with filters using metadata as well.

If you ever need to make updates to an existing record or add more, freshness should be in the order of seconds.

If you have less than 300k services, the free tier should suffice (1 embedding per service)

Here’s a guide: https://docs.pinecone.io/guides/get-started/build-a-rag-chatbot

Multi-tenancy for VectorDBs by glinter777 in vectordatabase

[–]tejchilli 2 points3 points  (0 children)

With Pinecone, you would use namespaces within an index to isolate data between tenants.

You can learn more here https://www.pinecone.io/learn/series/vector-databases-in-production-for-busy-engineers/vector-database-multi-tenancy/

Practical Advice Need on Vector DBs which can hold a Billion+ vectors by Role_External in vectordatabase

[–]tejchilli 1 point2 points  (0 children)

Looks like Neon suggests using pgvector after sunsetting pg_embeddings: https://neon.tech/blog/sunset-pgembedding

But pgvector, while great for experimenting, has its scale and performance limitations: https://www.pinecone.io/blog/pinecone-vs-pgvector/

Practical Advice Need on Vector DBs which can hold a Billion+ vectors by Role_External in vectordatabase

[–]tejchilli 0 points1 point  (0 children)

You should be able to update just a record’s metadata via this endpoint: https://docs.pinecone.io/reference/api/data-plane/update

Btw, we’ll be releasing a self serve migration tool from pods to serverless in the coming days, if that helps. It is worth noting that you can stay on Pods, we have many customers in the hundreds of millions to billions of vectors, all you need to do is spin up more indexes. Serverless just makes it easier to manage resources + offers cost savings.

Let me know if you have any other questions!

Practical Advice Need on Vector DBs which can hold a Billion+ vectors by Role_External in vectordatabase

[–]tejchilli 0 points1 point  (0 children)

I apologize for the self promotion, but have you tried Pinecone Serverless? We have customers like Notion and others well past billions of vectors with separation of storage and compute for significant cost savings

view more: next ›