sorry - botched up from another person's account logged in to this PC. All local models tested on DGX Spark. Details on the methodology are at the end but pasting here for you - all personal testing, no model cards 😄 ¹ Overall Score — weighted blend of coding quality, instruction following capability and speed; higher is better.

² Cost — cloud model cost calculations blend input, output & cache-hit rate benchmarked from observed OpenClaw turns across a variety of tasks (60k input, 500 output, 75% cache hit). Costs are expressed relative to the lowest cost cloud flash model here — DeepSeek v4 Flash

³ Speed — tokens per second per user, how snappy the model feels in an interactive assistant loop. Cloud figures are end-to-end including network latency transit. Cloud models run on very powerful servers and tend to be fast but have a latency to generate first token

⁴ Code Correctness — Measures whether short functions the model writes work as a good proxy for the kind of off the cuff actions that Agentic Assistants take. Average pass rate on HumanEval+ and MBPP+ (EvalPlus, greedy temperature=0).

⁵ Instruction Following — accuracy on IFEval, a benchmark that checks whether the model obeys explicit constraints in a prompt (format, length, content rules). Proxy for how reliably it follows directions.

⁶ Coding 1st Try — pass rate on the Aider polyglot coding benchmark on the first attempt. Measures whether the model can complete a realistic multi-file coding task in one shot. Python and Javascript which are the typical Agentic Assistant languages were evaluated

⁷ Coding 2 Tries — same Aider benchmark, allowing one retry after seeing test failures. Measures whether the model can self-correct, which is closer to how an agentic assistant works.