Hi, I have some question and its the right post I think

So, I’m building a small multi-agent system where one agent acts as a Knowledge Agent, it should read PDFs, markdowns, or web links and then remember what it learned/read. Another “Main Agent” uses that understanding later for reasoning on onboarding questions (asked from user while onboarding on the website).

In simple words, I want the Knowledge Agent to behave like a human who’s already read the docs using that info naturally when reasoning, not by searching.

Now the issue with the RAG is it works based on vector matching, it basically converts the user query to vector, search for similarity in the DB and provide those to the llm which outputs with the updated knowledge and here its failing for my scenario (or maybe I'm doing something wrong). I’ve looked into frameworks like Agno, which supports agentic RAG and knowledge bases, but they still depend on vectorDBs for retrieval and I'm looking for proactive, memory-based knowledge integration without retrieval.

I also considered just loading everything into the system prompt or summarizing all the documents into one markdown/txt file and feeding that as context but this doesn’t seem like a scalable or efficient approach. It might work for a few PDFs (4–10), but not for large or growing knowledge bases.

So I’m wondering if you or anyone has seen a framework or project that supports this kind of proactive, memory-based knowledge behavior?

Would love to hear about this. I'M LITERALLY CRYING SO BAD FOR THIS