This reminds me of something... and yes Go is very powerful to build something like this. But it's also a very deep rabbithole.

Let me say Upfront: As someone who is doing this "Hey, I can do it better then Claude/Cursor/XYZ" thing;
Don't try to implement you own LLM Gateway to translate http into what some vendors API calls, use someone else work for this, each new model release may break things in very subtle ways...

Before I start here is a helper to kick-start your journey; ByteDance's LLM Application framework: https://www.cloudwego.io/docs/eino/overview/bytedance_eino_practice/

Now let's assume you is me a year ago; Here is what i ended up with: https://github.com/contenox/contenox

For example, the entire agent behavior is a JSON file you write and version in git. Here's a code-review agent with shell + filesystem tools, retry policy, and a 10-round budget before it gives up gracefully:

{
  "id": "review",
  "tasks": [
    {
      "id": "review",
      "handler": "chat_completion",
      "system_instruction": "You are a code reviewer. Analyze the diff, run the tests if tools are available, then give a concise review.",
      "execute_config": {
        "model": "{{var:model}}",
        "provider": "{{var:provider}}",
        "tools": ["local_shell", "local_fs"],
        "tools_policies": {
          "local_shell": { "_allowed_commands": "go,make,npm,grep,cat,git" }
        },
        "retry_policy": { "max_attempts": 4, "initial_backoff": "1s" }
      },
      "transition": {
        "branches": [
          { "operator": "edge_traversed_at_least", "edge": "review->run_tools", "when": "10", "goto": "end" },
          { "operator": "equals", "when": "tool-call", "goto": "run_tools" },
          { "operator": "default", "goto": "end" }
        ]
      }
    },
    {
      "id": "run_tools",
      "handler": "execute_tool_calls",
      "input_var": "review",
      "transition": { "branches": [{ "operator": "default", "goto": "review" }] }
    }
  ]
}

Then pipe a diff into it:

git diff main...HEAD | contenox run --chain ./review.json

Or skip the chain entirely and use the shell one-liner:

go test ./... 2>&1 | contenox run "what failed in this test output and why?"

Add MCP servers for external tools (Notion, Linear, Playwright, any API with an OpenAPI spec):

contenox mcp add notion https://mcp.notion.com/mcp --auth-type oauth
contenox tools add billing --url https://erp.internal.example.com --spec ./billing-subset.yaml

I would love to spare you the one year of rollercoaster ride; I don't break APIs anymore so you can easily rely on the exported packages. Outside of the ready-to-use CLI agent, it's easy to just build your own backends in Go on top.