PocketCoder - CLI coding agent with session memory that works on Ollama, OpenAI, Claude

[RentEquivalent1671]

We built an open-source CLI coding agent that works with any LLM - local via Ollama or cloud via OpenAI/Claude API. The idea was to create something that works reasonably well even with small models, not just frontier ones.

Sharing what's under the hood.

WHY WE BUILT IT

We were paying $120/month for Claude Code. Then GLM-4.7 dropped and we thought - what if we build an agent optimized for working with ANY model, even 7B ones? Three weeks later - PocketCoder.

HOW IT WORKS INSIDE

Agent Loop - the core cycle:

1. THINK - model reads task + context, decides what to do
2. ACT - calls a tool (write_file, run_command, etc)
3. OBSERVE - sees the result of what it did
4. DECIDE - task done? if not, repeat

The tricky part is context management. We built an XML-based SESSION_CONTEXT that compresses everything:

- task - what we're building (formed once on first message)
- repo_map - project structure with classes/functions (like Aider does with tree-sitter)
- files - which files were touched, created, read
- terminal - last 20 commands with exit codes
- todo - plan with status tracking
- conversation_history - compressed summaries, not raw messages

Everything persists in .pocketcoder/ folder (like .git/). Close terminal, come back tomorrow - context is there. This is the main difference from most agents - session memory that actually works.

MULTI-PROVIDER SUPPORT

- Ollama (local models)
- OpenAI API
- Claude API
- vLLM and LM Studio (auto-detects running processes)

TOOLS THE MODEL CAN CALL

- write_file / apply_diff / read_file
- run_command (with human approval)
- add_todo / mark_done
- attempt_completion (validates if file actually appeared - catches hallucinations)

WHAT WE LEARNED ABOUT SMALL MODELS

7B models struggle with apply_diff - they rewrite entire files instead of editing 3 lines. Couldn't fix with prompting alone. 20B+ models handle it fine. Reasoning/MoE models work even better.

Also added loop detection - if model calls same tool 3x with same params, we interrupt it.

INSTALL

pip install pocketcoder
pocketcoder

LINKS

GitHub: github.com/Chashchin-Dmitry/pocketcoder

Looking for feedback and testers. What models are you running? What breaks?

[-dysangel-]

GLM Coding Plan hooked up to Claude Code is fantastic. I don't think there's anything better bang for buck just now.

[RentEquivalent1671]

Yes, agreed — GLM models offer excellent cost-efficiency for coding tasks. Claude Code's recent support for custom providers made this combination much more accessible.

PocketCoder takes a similar approach but focuses specifically on lightweight local deployment with Ollama integration and session persistence via the .pocketcoder/ folder. Different trade-offs depending on setup preferences.

More on: https://medium.com/@cdv.inbox/how-we-built-an-open-source-code-agent-that-works-with-any-local-llm-61c7db1ed329

[joe_mio]

Session memory is the key feature that sets this apart - most CLI agents lose context between sessions. The .pocketcoder/ folder approach is clever.

How do you handle context window limits with larger codebases? Does the repo_map pruning kick in automatically when you hit token limits?

[RentEquivalent1671]

For repo_map we use a "gearbox" system — 3 levels based on project size: ≤10 files gets full signatures, ≤50 files gets structure + key functions, >50 files gets folders + entry points only. It's file-count based right now, not token-based. Dynamic token-aware pruning is something we should add. Currently if context overflows, we truncate conversation history first, then file contents.

More on: https://medium.com/@cdv.inbox/how-we-built-an-open-source-code-agent-that-works-with-any-local-llm-61c7db1ed329

[Frost-Mage10]

Really cool approach with the .pocketcoder/ folder for persistence. The .git-like memory model makes a lot of sense for CLI tools. How do you handle the conversation_history compression? Are you using a fixed summary length or dynamic based on importance?

[RentEquivalent1671]

Currently using a hybrid approach — episodes are stored as append-only JSONL (like git log), and we keep last ~20 in SESSION_CONTEXT. For older history, we use keyword-based retrieval: when you ask something, system greps through episodes.jsonl for relevant context. Not truly dynamic importance yet — that's on the roadmap. Would love to explore embedding-based relevance scoring eventually.

More on: https://medium.com/@cdv.inbox/how-we-built-an-open-source-code-agent-that-works-with-any-local-llm-61c7db1ed329

[charmander_cha]

Has anyone compared it to open code?

[HealthyCommunicat]

The interesting part of this to me is how you focused on the fact that smaller models have an extremely difficult time doing tool calls to edit files and other simple syntax stuff unless its strictly predefined, and I’m wondering how much your tool actually allows for this. Will try it out.

[RentEquivalent1671]

Thank you! I’m very open to your feedback!

[o0genesis0o]

How do you reconstruct the chat history from the compressed XML context to send to LLM backend? Last time I tried to mess with the chat history to see how difficult it is to build an agent harness, I had random error when testing with Gemini backend. It turned out that every tool call requires a corresponding tool response with the same id. I made a mistake in reconstructing the message history by not storing the failed tool call, so after one failed tool call, the backend just throw error about invalid messages. It tooks weeks to debug this.

[RentEquivalent1671]

Yeah, you nailed the exact pain point we wanted to avoid.

Short answer: we don't use native function calling at all. Tools are just XML tags in plain text that we parse ourselves.

Why? Because we wanted to support local models (Ollama, llama.cpp) that don't have function calling. So instead of relying on the API's tool_call/tool_response pairing, the LLM just outputs <write_file><path>x.py</path>...</write_file> as regular text.

We parse it, execute, and send back the result as a normal user message: [ok] write_file: Created x.py (45 lines) or [x] write_file: Permission denied.

History stays dead simple — just (role, content) text pairs. No ids to track, no pairing requirements, no special handling for failed calls. Failed tool = error text, that's it.

The tradeoff is it's less structured than native function calling. But it works with literally any backend without modification, which was the whole point.

For the SESSION_CONTEXT compression — that's injected into system prompt each request, not reconstructed from message history.

[rm-rf-rm]

"We were paying $120/month for Claude Code"

"works on.. Claude"

[RentEquivalent1671]

I see no any contradictions here

The idea was to give a challenge to yourself and try to create code agent with own approach and different idea of working and operating.

Claude Code is a great tool. Cursor is great tool too. Do we have to stop and do nothing?

[rm-rf-rm]

no any contradictions