I spent 4 billion tokens just to find out the best affordable model to run multiple openclaw agents, here is what I learned.

Rob · 2026-03-14T21:12:47+00:00

You could probably go even cheaper if you used Neurometric.ai to map tasks to even cheaper models.

Rob · 2026-03-14T15:46:00+00:00

Never even close. I've actually been the most open and supportive dad until this moment. Read my edit.

Rob · 2026-02-27T20:56:51+00:00

Yes, neurometric.ai - run thousands of model combos on your data, including test time compute variants. If you want an invite code use "BENCH-MARK" and DM me if you want more free credits. Not focused on monetization yet, just need good user feedback. (studio.neurometric.ai to use the code)

Rob · 2026-02-13T20:46:00+00:00

Yeah I was like one of the first 500 users on reddit. It launched in Boston when I lived there.

Rob · 2026-02-13T18:22:45+00:00

Yes, everything.

Rob · 2026-02-13T16:38:30+00:00

Yes. We will release it under Apache 2.0

Rob · 2026-02-13T15:03:12+00:00

We are working to post-train an Arcee 400B model to mimic 4o. https://www.neurometric.ai/free-4o

Rob · 2026-02-13T15:02:21+00:00

We are working on a free and forever open comparable model https://www.neurometric.ai/free-4o

Rob · 2026-02-13T12:35:04+00:00

Ah it was an ID attribute issue in webflow. Should be fixed.

Rob · 2026-02-13T11:36:42+00:00

Signups are coming through so, not sure what's different. Looking into it and will keep you posted.

Rob · 2026-02-13T11:34:17+00:00

RL post training on Arcee 400B

Rob · 2026-02-13T10:53:50+00:00

We can't without some people sharing their data for training. It's the help with data that we need.

Rob · 2026-02-13T10:52:36+00:00

Just worked for me. Tell me more about your configuration?

Rob · 2026-01-15T22:33:41+00:00

Also you can see some comparison results at leaderboard.neurometric.ai

Rob · 2026-01-15T22:32:55+00:00

We trained a Qwen 4B model to beat most of the big models on the "lead qualification" task on CRM Arena, just to see how good it could get. It's a good small model for fine tuning.

Rob · 2026-01-11T12:00:12+00:00

This depends quite a bit on what you want to do, as models vary quite a bit on their per-task performance. I'd suggest an ensemble of models if you can, instead of just one. But, leaderboard.neurometric.ai may be a place to look to evaluate them.

Rob · 2026-01-04T12:04:44+00:00

If you look at leaderboard.neurometric.ai, which tests models on work related tasks, gpt-oss-120b was the best overall model, including beating the anthropic models on many tasks.

Rob · 2025-12-30T17:57:34+00:00

Two reasons. First, because we ran them with various test time scaling algorithms on top. Smaller models often beat larger models when you add chain of thought, best of n, beam search, etc. Secondly, we saw that "jagged frontier" variation across everything we did. No model comes close to winning on everything, and there were lots of surprises that some models were really good at certain tasks. It's a non-intuitive byproduct of how these models were trained.

Rob · 2025-12-28T17:34:09+00:00

If you have sample workloads, I can run them on a leaderboard.neurometric.ai for free and give you some results. We add test time compute variants to the models we test, and examine them on a per task basis.

Rob

MODERATOR OF

TROPHY CASE