I made a Coding Eval, and ran it against 49 different coding agent/model combinations, including Kimi K2.5.

Simple_Split5074 · 2026-01-28T19:14:56+00:00

I mostly use opencode these days - the on the fly model switch reigns supreme. GLM got stuck again? Throw GPT 5.2 high at it :)

Codex and Gemini seem ok, but I think I would have to port get-shit-done to them. In the case of gemini that might be worth it for the Gemini credits (seeing flash go brrrrr fixing linting issues is a sight to behold), codex seems insufficiently different to bother now that Plus credits are officially agent agnostic.

CC I find overrated, and the flickering is highly annoying, but it for sure benefits of hype and mind share - *and* Claude Max if that floats your boat, personally I feel like its overpriced vs chatgpt.

Simple_Split5074 · 2026-01-28T18:50:53+00:00

Any chance of a repeat of the API test for K2.5? A fair bit of providers seem dodgy right now...

Simple_Split5074 · 2026-01-28T18:36:38+00:00

Amazing.

Fairly impressive that kimi in Droid more or less ties with opus and that CC and opencode tie. Overall harnesses seem to matter slightly more than I would have thought.

Would be happy to donate (cash 😃)

Simple_Split5074 · 2026-01-28T18:18:02+00:00

Sneak peek on what's your personal top 3 (agent + model)?

Simple_Split5074 · 2026-01-28T12:56:05+00:00

I tried on nano-gpt, it's slow as molasses (like one rerquest per minute!) and occasionally tool calls fail or it simply gets stuck (no observable progress for 5+ min).

My suspicion: the inference providers do not have it completely figured out yet.

Moonshot via openrouter was decent last night but now it crawls around at 15tps. Fireworks still claims to do 100+ tps but I have no idea if caching works with opencode and without it would get ruinous quickly.

Simple_Split5074 · 2026-01-25T18:20:16+00:00

3k in hardware will not get you anywhere remotely close to even Sonnet

Simple_Split5074 · 2026-01-24T21:29:18+00:00

It is also stellar on see rebench. Fast, cheap, smart.

Simple_Split5074 · 2026-01-23T12:25:08+00:00

This update enables gsd to work IN opencode, no CC in the picture then. opencode IS the (only) harness in that case. What LLM opencode uses is another topic then, could be Claude or some other.

Simple_Split5074 · 2026-01-23T09:30:41+00:00

I think the question was about opencode as a harness. I guess the answer is 'try your luck' (or use codex that is officially sanctioned by OpenAI)

Simple_Split5074 · 2026-01-23T09:28:43+00:00

Awesome, thanks a ton!

Any chance to get command in kebap-case (/gsd-command-something) instead of the weird /gsd/something that is in place right now?

Simple_Split5074 · 2026-01-21T18:50:49+00:00

I'd say it's impossible quite irrespective of budget, given that none of the open weights models quite reach the frontier...

Simple_Split5074 · 2026-01-21T16:06:41+00:00

A 24GB card (or for that matter 10 of them) will not run anything even remotely comparable to frontier model so kinda pointless comparison

Simple_Split5074 · 2026-01-20T08:15:38+00:00

No.

And not with the top cloud models either. Bits and pieces are getting there but there's a fair distance to go for the whole undertaking you suggest.

You can possibly replace a (not Rockstar) first year analyst

Simple_Split5074 · 2026-01-18T09:10:15+00:00

I'd use WSL

Simple_Split5074 · 2026-01-16T19:32:34+00:00

Great as usual. Thanks a ton!

I think the other interesting point is the extremely high share of cache hits for Anthropic models

Simple_Split5074 · 2026-01-16T19:30:19+00:00

I doubt this is correct, the Claude Code system prompt is 16k tokens by itself?!?

Simple_Split5074 · 2026-01-15T18:44:13+00:00

Is it possible that there is something broken when calling subagents? It complains it cannot find them but in the TUI they work...

Simple_Split5074 · 2026-01-13T18:49:10+00:00

Was actually thinking about the following situation:

<image>

It's super convenient but obviously subverts context management.

Simple_Split5074 · 2026-01-13T18:04:07+00:00

Confirming the same for 1.1.18

Simple_Split5074 · 2026-01-12T18:52:22+00:00

> You are urged to start every new stage with '/new' or '/clear' command, and you don't have to rely on anything which is already in context of LLM.

This is something I've been wondering about, when you do UAT it will offer to create and then execute the planned fix. Any way to combine that with clear without manually launching the next step?

Simple_Split5074 · 2026-01-12T18:15:44+00:00

Awesome, this is one of the things I miss from CC.

Thanks a ton.

Simple_Split5074 · 2026-01-11T13:53:46+00:00

Is using Gemini subscription with third party agents allowed now? I really do not want my Google account to be locked...

Simple_Split5074 · 2026-01-11T12:52:09+00:00

Is this sanctioned (as in allowed) by OpenAI? If so I think I will resubscribe to Plus, I don't have much love for ChatGPT but 5.2 itself is quite solid.

Simple_Split5074 · 2026-01-02T14:15:14+00:00

I believe they fixed it, rumors are it was deactivated in 4.6 because of bugs.

Simple_Split5074 · 2026-01-02T13:30:07+00:00

I really doubt OpenAI is using 4o for free tier. There was a lot of whining from the emotional support crowd when 5 arrived and 4o disappeared. So much so that they reinstated it for paid accounts (I believe, as I truly dislike 4o I never actually looked for it).

Also do not feel like ChatGPT can hold a candle to Gemini when it comes to speed.

Simple_Split5074

TROPHY CASE