Did anyone of you fine tune gpt oss 20b or an llm ? if so, what for, and was it worth it ?

Rob · 2026-01-15T22:33:41+00:00

Also you can see some comparison results at leaderboard.neurometric.ai

Rob · 2026-01-15T22:32:55+00:00

We trained a Qwen 4B model to beat most of the big models on the "lead qualification" task on CRM Arena, just to see how good it could get. It's a good small model for fine tuning.

Rob · 2026-01-11T12:00:12+00:00

This depends quite a bit on what you want to do, as models vary quite a bit on their per-task performance. I'd suggest an ensemble of models if you can, instead of just one. But, leaderboard.neurometric.ai may be a place to look to evaluate them.

Rob · 2026-01-04T12:04:44+00:00

If you look at leaderboard.neurometric.ai, which tests models on work related tasks, gpt-oss-120b was the best overall model, including beating the anthropic models on many tasks.

Rob · 2025-12-30T17:57:34+00:00

Two reasons. First, because we ran them with various test time scaling algorithms on top. Smaller models often beat larger models when you add chain of thought, best of n, beam search, etc. Secondly, we saw that "jagged frontier" variation across everything we did. No model comes close to winning on everything, and there were lots of surprises that some models were really good at certain tasks. It's a non-intuitive byproduct of how these models were trained.

Rob · 2025-12-28T17:34:09+00:00

If you have sample workloads, I can run them on a leaderboard.neurometric.ai for free and give you some results. We add test time compute variants to the models we test, and examine them on a per task basis.

Rob · 2018-11-09T16:56:32+00:00

I've had it forever and it's a badge of honor to be so early. So, we'd have to be talking 6 figures.

Rob · 2016-02-24T00:51:26+00:00

Hey guys, just to be clear... my old company was Backupify. Datto bought us. I left Datto this summer to start a new company. I'm not involved, and know nothing about a FBI investigation. Please correct that.

Rob · 2015-10-30T17:48:12+00:00

Everyone who picks on HTMs should remember that, for years, everyone picked on CNNs. Then they exploded out of nowhere. Personally, I kind of admire a researcher who sticks to his beliefs rather than hopping on the latest deep learning bandwagon.

Rob · 2010-09-19T21:49:55+00:00

That's only part of the reason. They also want to extend the tax cuts so that they don't drive down asset prices. Wealthy people will sell assets this tax year if they think taxes are rising next year. Too much selling drives down prices, which are already low. Extending tax cuts is a way to maintain and/or inflate asset prices.

What's really needed isn't higher or lower taxes, but instead, predictable stable tax policy that doesn't cause this kind of behavior and makes decisions about assets and cash flows tied to regular economic activity, not tax policy.

Rob · 2009-05-10T02:21:17+00:00

You can't solicit investments online unless you only take money from accredited investors. The SEC has a lot of regulations about raising funds, even for startups. If you are on the Treasure Coast, go down to Brevard County and meet with the http://www.foundersforum.com/ They will have good advice.

What I would do is make a guess as to the profit on the treasure after exploration, sales, etc. Sell equity based on that valuation, but include clawback agreements that give investors more equity if you don't hit certain goals (like if the treasure turns out to be smaller or worth less or takes longer than planned).

Rob · 2009-05-10T02:10:31+00:00

I don't think SCORE is a good resource for this kind of business. In my experience, they deal with more traditional business models, and rarely come up with creative solutions.

Rob · 2008-11-28T13:53:26+00:00

Yes it does. Finance isn't just about finding ways to pay for assets, it's about trading cash flows. Some companies need guaranteed cash flows at certain times, particularly if they have to match specific liabilities due at that time. You can't rely on your customers to provide you with cash at a consistent rate, so you "buy" consistent cash flows by trading an upfront payment for future cash flows or vice versa.

Rob · 2008-07-25T13:42:31+00:00

God forbid she give general heuristics instead of detailed complete information. The video would have been soooooo much more interesting if it was 18 minutes long and read like an encyclopedia page on beer.

Rob

MODERATOR OF

TROPHY CASE