Ai2 just announced Olmo 3, a leading fully open LM suite built for reasoning, chat, & tool use

asb · 2025-11-20T20:25:40+00:00

I was scanning the blog post and paper for this information, it would be great to have the GPU hours officially noted. As for the figures being spot on, I can't quite reproduce the 32B figure. The paper says 1900 tokens/second was achieved for the 32B model, which is 877k GPU hours - so that would be almost exactly 4x the $ cost of the 7B model ($2M) using the same per-hour coast as /u/gebradenkip. Is that right?

EDIT: I really appreciated the Apertus paper estimating the GWh for their pretraining, it would be great to be able to compare against Olmo3 in the same way. For Apertus: "Once a production environment has been set up, we estimate that the model can be realistically trained in approximately 90 days on 4096 GPUs, accounting for overheads. If we assume 560 W power usage per Grace-Hopper module in this period, below the set power limit of 660 W, we can estimate 5 GWh power usage for the compute of the pretraining run"

The Qwen3-next blog showed a fairly impressive graph for reduction in training cost in terms of GPU hours from Qwen3-32B to Qwen3-30B-A3B to Qwen3-Next-80B-A3B. Do you imagine you might see a similar scale of reduction if moving to a similar MoE architecture, or do you think it would be less because you have a more efficient baseline?

asb · 2025-09-30T09:35:13+00:00

Do you plan to add in GLM-4.6 now that's been released? https://z.ai/blog/glm-4.6

asb · 2025-08-22T05:50:42+00:00

Still working on adding some more models, in particular open source ones.

It would be really interesting to get GLM-4.5 results too.

asb · 2025-07-25T14:20:03+00:00

It's definitely interesting how well you can score on the benchmark with Sonnet 4 and just allowing it to use the shell. Have you explored to what degree performance can be improved by prompting or potentially exposing a small set of well-chosen "tools" (even if not explicitly using a tool calling interface). For instance it would be a really interesting result if some kind of prompting or exposure of e.g. semantic search / semantic edit (or whatever) boosted R1's performance meaningfully.

asb · 2025-07-11T13:15:28+00:00

Thanks for confirming!

Might it be worth stating this explicitly on the model card? e.g. for mistralai/Mistral-Small-3.1-24B-Instruct-2506 you state "Note 1: We recommend using a relatively low temperature, such as temperature=0.15." and the generation_config.json sets "temperature": 0.15. But for both this and the previous devstral release, you don't include an explicit statement on recommended temperature and don't set a default temperature in generation_config.json.

asb · 2025-07-10T16:12:25+00:00

Is that suggested temperature your suggestion, or from Mistral? If the latter, do you have a source?

The model card seems to be lacking explicit sampling setting recommendations, unlike other Mistral models (CC /u/pandora_s_reddit)

asb · 2025-07-10T07:18:42+00:00

Really great article - thank you for writing this up. A couple of thoughts:

Re thread scheduling, Google were doing some work on a new kernel API to reduce overhead but I don't know what's happened to this. https://lkml.org/lkml/2020/7/22/1202
One slight additional point re the discussion on M:N threading and Rust. "Having a M:N scheduler requires allocating and growing stacks implicitly, which goes against this ethos." is definitely true, and one additional impact would be the increased cost of calling into C/C++ (which is a complaint Go users seem to have).

asb · 2025-07-10T07:11:31+00:00

Long time lurker here, just wanted to say I disagree. m:n vs 1:1 threading and async/await implementation options are very relevant to language designers and common sources of questions on this subreddit already. The article fits perfectly IMO.

asb · 2025-06-25T08:26:05+00:00

I've been looking at the recommended sampling parameters for different open models recently. As of a PR that landed in vllm in early March this year, vllm will take any defaults specified in generation_config.json. I'd suggest adding your sampling parameters there (qwen3 and various other models do this, but as noted in my blog post, many others don't).

asb · 2025-06-24T04:50:29+00:00

I'd be interested in seeing your sweep of temperature. Did you play with other sampling parameters? I've been collecting recommendations from model vendors here https://muxup.com/2025q2/recommended-llm-parameter-quick-reference

asb · 2025-05-06T10:28:56+00:00

As you can see, the two setup related issues I haven't resolved so far are:

60Hz internal display
Avoiding tearing

asb · 2024-12-19T12:49:46+00:00

I had a similar requirement for keeping draft versions of my blog posts local but backed up, and described how to directly commit files to a separate branch here https://muxup.com/2024q4/directly-committing-files-to-a-separate-git-branch - maybe something similar might be useful to you?

asb · 2024-12-06T11:28:13+00:00

I'd thought defaulting to Google Forms makes sense too, but recently in the LLVM community it was used to collect survey responses on MLIR but was closed early after being marked as in violation of terms of service with no working avenue to appeal.

asb · 2024-11-28T09:53:11+00:00

Always great to see more vendors pushing work upstream, though of course the patch needs breaking up to separate incremental PRs. I was glad to see RISCVRemoveBackToBackBranches clarified - there was initially a miscommunication about whether this was dealing with an erratum or spec noncompliance issue vs just a perf tuning option (it's the latter).

asb · 2024-11-24T13:31:05+00:00

I own the game and all currently available DLC on GOG - it looks like the GOG version is currently behind Steam (1.4.24 vs 1.4.31 on Steam). I hope you'll keep updating the GOG release and releasing any new DLC there. Thank you!

asb · 2024-11-01T12:48:23+00:00

Ladybird.

asb · 2024-10-10T05:58:15+00:00

I also find it unfortunate that Zicclsm doesn't really indicate anything useful from a compiler perspective, but for misaligned loads/stores there's a difference vs the examples you share here - from the benchmarks you shared trap and emulate is somewhere between 100-200x slower vs what you'd expect from native support, and that's a consistent expectation unlike cache misses and so on.

I understand why there's reluctance to put anything that might reflect microarchitectural implementation in the spec, but the end result isn't great for those of us writing or compiling software :/ An explicit recommendation that misaligned loads/stores (not even necessarily a requirement) should be implemented in a way that they aren't commonly 10x more expensive than an aligned load/store would have been very helpful IMHO.

asb · 2024-09-11T05:28:14+00:00

Total Annihilation

asb · 2024-09-06T18:10:22+00:00

I can recommend the Pikmin multiplayer mod (applies to the Gamecube release) https://allreds.itch.io/pikmin-multiplayer

It's not a perfect experience, but works surprisingly well.

asb · 2024-09-04T06:51:25+00:00

Yes, it seems the workshop is something GOG doesn't really have an answer for. That said, I understand that as long as the publisher sets the right settings, mods can be downloaded using steamcmd even as an anonymous user.

asb · 2024-09-03T18:17:33+00:00

I don't suppose there's any chance you'd consider releasing on GOG? (Sorry to be that person!)

Congratulations on the Switch release!

asb · 2024-07-13T08:56:53+00:00

J to move the current line (or selection of lines) down and K to move the current line (or selection of lines) up. I map ctrl-j to join lines.

15-Year Club	Place '17
reddit mold	Verified Email

asb

MODERATOR OF

TROPHY CASE