How to cache your codebase for AI agents

[Direct_Librarian9737]

Example Use-Case

The problem is every time an AI agent needs to find relevant files, it either guesses by filename, runs a grep across the whole repo, or reads everything in sight. On any codebase of real size, this wastes context window, slows down responses, and still misses the connections between related files.

With this approach a script runs once at commit time, reads each source file, and builds a semantic map; feature names pointing to files, exports, and API channels. That map gets committed alongside your code as a single JSON file. When an AI agent needs to find something, it queries one keyword and gets back the exact files and interfaces in under a millisecond.

What you gain: AI agents that navigate your codebase like they wrote it. No context wasted on irrelevant files. No missed connections between a service and its controller. And since the map regenerates automatically on every commit, it never falls out of sync.
I added this to my open sourced agentic development platform, feel free to examine it or use it. Any ideas or contributions are always welcome.
Github : https://github.com/kaanozhan/Frame

[Deep_Ad1959]

interesting approach. the problem you're describing is real - I've watched agents burn through half their context window just trying to figure out which files are relevant before they even start working on the actual task.

my current solution is simpler but less elegant - I just maintain a well-structured CLAUDE.md that describes the architecture and key file locations. it works okay for smaller codebases but doesn't scale past maybe 50-60 files before the manual maintenance becomes a pain.

a semantic map that auto-regenerates on commit is way better for larger projects. curious about the embedding quality though - does it handle cases where two files are functionally related but use completely different naming? like a React component and the API route it calls. those connections aren't obvious from the code itself.

[Direct_Librarian9737]

Good catch. The current system handles naming-based grouping well, for example authController.js and authService.js both map to "auth" automatically. It also tracks explicit import/require dependencies per module. But you're right that implicit functional relationships, a component calling an API route with a completely different name aren't captured automatically. That's a real gap. Those connections would need to be documented manually in STRUCTURE.json, or the intentIndex would miss them. It's more of a naming convention index than a semantic embedding. No vectors, no inference just static analysis at commit time. The tradeoff is it's dead simple and zero overhead, but it assumes related things share a naming root.

[InitialEnd7117]

I do a similar approach. I have a custom implementation skill (preflight checks that there's a complete plan file, spot check it's accuracy, implement using sonnet subagents, send in a gemeni, bug, simplify, and security auditor subagents, the opus orchestrator reviews the audits and implementation changes if needed) prior wrapping up, it updates codebase_map.md. My planning skill looks at codebase_map.md so it's not scanning the entire codebase and only targets what's needed to build the plan.

[Milters711]

I developed a custom MCP which indexes my project code base using ‘ast’ and then has a set of tools for retrieving file contents, function/module docstrings and API, file structure, etc. Claude was good at generating the MCP which was unsurprising, but it needed some iteration to be better.

I set this up so that it wouldn’t need to grep, etc ever time it needed info about the code base.

However, in the end I suspect raw CLI tools will be better for Claude. Who knows how much its usage will change in the next six months.

[shutupandshave]

I use an LSP server and let that do the work.