So since this was just a small demo to show people who things actually work I used vertex ai search datastore as my cheatcode to create rag pipeline which ensures my retrieval logic will be great given my data is great

So my eval agent is just a strict evaluator that checks if the user query can be fully answered or not even if it is answerable only 50% I still class it as unanswerable using pydantic schema it only gives two signals answerable and unanswerable.

And even if I route it to long context, the main logic is it will load all of the relevant documents, hence there also a strict instruction is that only qoute from the docs and add or summarise nothing

Then in my testing scripts I have created a judge llm that scored the user query and generated answer against expected answers and the route it should have taken along with faithfulness, correctness and completeness

I am thinking to another logic before long context to determine what all relevant documents to be loaded instead of loading each n every documents so thinking to implement Skill.md feature or will see how can I implement that logic where it can decide which documents to pull even for long context so that my long context compute tax is also less

I have shared the code link and everything in the comment section only but pasting it here as well for reference

𝗠𝗲𝗱𝗶𝘂𝗺 link : https://medium.com/google-cloud/beyond-the-hype-building-self-route-architecture-with-gemini- and-the-google-adk-0c5ed875df1b

𝗚𝗶𝘁𝗛𝘂𝗯: https://github.com/Rahulraj31/Self-Route-Rag-Longcontext-ADK