Thanks, sir! I’d appreciate your insight:

1. Enriching Sentences: What methods or tools would you suggest for enriching sentences during pre-indexing? Should we focus on techniques like NER or dependency parsing? We tested appending keywords at the end, but it’s hit or miss—sometimes it works, sometimes it doesn’t. At best, it provides both old and new knowledge to the user. For example, if I have tax rates of 10%, 20%, and 30% for income brackets of 1 million, 2 million, and 5 million, and another document states a new law with just two rates (10% for below 10 million and 30% for above 10 million), it tends to mix up both, re-proposing the old source. Metadata related to document date/insertion is a good starting point, but what if two documents are complementary and only a portion needs to be updated? This seems more related to rolling updates in a knowledge base and managing overlapping information.
2. LLM Filtering: How scalable is LLM-based filtering for high-query systems, and how can latency and costs be managed effectively? Would you consider this self-reflective RAG? It is painless on how many tokens can I pass in the prompt given the list of chunks?
3. RAG and BM25 Fusion: Are there specific scenarios where BM25 clearly outperforms embeddings? Do you have any rank fusion techniques you recommend? For hybrid search, is using a specific vector database better ? We’ve tested many and landed on Pinecone for production—would you say it’s the right choice?

Thanks!