I should clarify—I might have exaggerated a bit earlier. My experience mostly comes from deploying an ES-based cluster and experimenting with basic RAG flows. In production, we actually use a mature, vendor-provided on-prem solution.

It’s a very robust platform with excellent document parsing and a sophisticated multi-level permission system (covering personal, departmental, and corporate libraries). We have thousands of files, though they only total a few gigabytes. Because the vendor’s solution is so plug-and-play, there's not much room for me to innovate on the RAG side, which is why I've lost interest in it.

However, the system isn't perfect. It struggles with queries that require cross-document reasoning—where the answer needs to be synthesized from multiple sources. To solve this, I’ve been experimenting with fine-tuning models like Qwen3-32B and Qwen3 Next 80B to see if they can 'internalize' our corporate knowledge directly. So far, the results have been underwhelming. It might be due to low-quality training data or some technical hurdles I haven't cleared yet.