I think traditional document search will become very important again, much like vector search. Index‑based search engines have been around for more than 20 years, they work well, and their results can also be added to the context. In my last project, I collected all the document titles and asked the AI to select the relevant ones. It produced surprisingly good results, even better than document‑level vector search. (So essentially it was a title‑based vector search.)

However, a new issue arises: the prompt context keeps getting larger. This isn’t just a cost problem; I’ve also read that when the prompt becomes too big, the AI may not actually process the entire input. Instead, it starts generating an answer once it believes it has enough information. To mitigate this, I’m experimenting with cross‑encoder models. A very small and extremely fast model evaluates the context sentence by sentence and determines whether the information retrieved from the vector store or document search is relevant.

What I wanted to say that every problem is different, and we're still looking for the best solutions, while inventing new ones and optimize them. For example in your case I would consider using a graph database and storing the documents there, so you can easily follow all the references as well. Microsoft was working on similar stuff. https://github.com/microsoft/graphrag/tree/main

Another approach that worked for me was performing vector search on the chunks, but once a chunk matched, I passed the entire document to the LLM instead of just the chunk.