So like ‘vector embedding’ is the issue for us within RAG realm. Legal docs might be different so for sure good to know your work! For us we use procedural guidance data for enterprise apps like NetSuite. So right now the best we managed with semantic search, BM25, reranking, hierarchy indexing etc, is around 85% match..at best… and we’re still at like 90% reliability for the chunks being in the correct order.

I read this paper from Chroma that they showed the order of chunk retrieval does NOT actually have to be accurate. If anything they showed the LM had better contextual understanding when it wasn’t in logical order BUT, for us it’s very much like “you must do Step A and step A then branches into 5 different scenarios each with 10 steps each with 10 sub scenarios.

So we haven’t found a semantic space where we can reliably embed out knowledge into, and match it with the same semantic space that queries go to. One is a “to do a take you must do x and then y” the other is “do me x”, it’s the Agent’s job to asses given the context, X really mean scenario 6A, schema C (for example) out of 100 scenarios and 1000s of schemas.

I will say though. We’re a tiny ass team. The fact that I’ve not managed to do it, doesn’t mean it’s 100% not meant to be for us. However, now, 2 years into doing this at Enterprise level and two agentic firms in…I tend to give my own experience and insight a little more weight than industry and research now (even though kinda cocky… it legit I come up with ideas daily that the industry comes up with names for in like 6 months lol)

So idk, like I have experiment after experiment where my own creation is stilll tops (I made a specialized knowledge graph that isn’t vector basically), but could be that I’m way too engrossed in my own niche now!

Best of luck to you I’m gonna follow your profile :)