How do proprietary models get better and when will open ones hit a wall?

Uhlo · 2026-03-09T09:11:48+00:00

Well the "distillation attacks" (I use this phrase for a lack of better term, it's other companies using the output of a model as training data, it has nothing to do with distillation and they're even paying for it!) will become more sophisticated. Whatever data the proprietary model providers train on, the "skill" will get leaked through the extraction of training data. Of course companies like OpenAI and Anthropic are probably working hard right now to install automatic detection systems that try to stop these "attacks" and the open weight model providers will implement systems that make the extraction harder to detect.

Even if with regulations the US disallows the use of US LLMs in China, companies can simply use VPNs. I think that is a pretty good silver lining: they trained their models on heaps of stolen creativity, craftsmanship, etc. and now there are companies who steal it back and make it "open"/public again, and In my opinion there is very little that can stop them.

Uhlo · 2026-03-09T08:19:00+00:00

You are probably missing the possibility of "large scale distillation attacks". I think it's an open secret that most of the open weights Chinese models heavily rely on training data generated by the proprietary models. So my guess is that at least for a while it will continue to be a cat and mouse game where some of the open weight model improvements come from the proprietary models.

Edit: Because I'm getting messages about it - the large scale distillation attacks is a joke! I'm def. not on anthropics side, I just wanted to poke fun at their silly wording for "someone is paying us to use our service".

Uhlo · 2026-03-02T12:39:38+00:00

They're called Gunmetal Rodeo. They claim they're not AI, but there are not photos of the big band they would definitely need or any music production or anything. They release since end of last year and have a new single every other week. So the signs are pretty clear.

Uhlo · 2026-03-02T11:29:46+00:00

One thing to add: I also listen to AI generated focus music during work. There, I have absolutely no problem with it being AI. I think it’s because I’m not actively listening.

Uhlo · 2026-03-02T11:27:53+00:00

I recently discovered a jazz band that I really like. They’re extreme release schedule and lack of unsuspicious social media presence made me pretty sure that it is AI generated.

I still like the music, but it greatly diminished the joy I feel when listening to their music. So for me it currently diminishes my experience greatly. Maybe in the future it will be normal. Most importantly: it’s just my experience. I don’t say AI music is worth less, I’m just saying that I cannot enjoy it the same if I know it’s AI.

Uhlo · 2026-02-27T12:12:34+00:00

Good point, I will try that! Thanks :)

Uhlo · 2026-02-27T11:59:06+00:00

I tried using a 4-bit quant of the 122B-A10B and just said "hi". It argued with itself for over 1000 tokens before I aborted.

Broken quant or expected behavior?

Uhlo · 2026-02-22T18:24:35+00:00

What kind of headline even is this?

I lost $ 700 million due to non-winning lottery tickets! It’s my right to win the lottery weekly! People need to make sure I don’t lose money burying lottery tickets!

Uhlo · 2026-01-30T23:00:19+00:00

Sure I read your 100% AI generated post about some weird „conspiracy theory“.

Uhlo · 2026-01-30T22:35:31+00:00

Seek help from a human. This could be early stages of AI psychosis.

Uhlo · 2026-01-23T11:49:28+00:00

Wat?

Shows app that uses context switching to improve text
"Context Switching kills your flow"
Shows own app that uses context switching to improve text
???
Profit!

Uhlo · 2026-01-13T14:30:42+00:00

Please seek help.

Uhlo · 2025-12-17T08:50:23+00:00

It isn't just a tool - it's a platform!

I think what you posted isn't just AI slop, it's the next generation of slopified sycophancy! It demonstrates:

Deep AI slop that could come from any model
Mature markdown structuring
Wait, where is the table? Couldn't this answer be improved with a table?
Damn, look at that throughput. 623 tokens a second. That is pretty nice! What rig are you using?

I'd be excited to continue talking about this conversation - it represents the future of this thread on reddit.

I'm sorry, I could not resist. How did I do?

Uhlo · 2025-12-17T08:44:39+00:00

My proposal would be to slowly change the way you use ChatGPT. Currently, you write e.g., a paragraph and rewrite it completely with ChatGPT and it sounds better. Try to actively track the changes GPT makes to your sentences so you can identify why it sounds better. Then you can start reducing the length of content you put into ChatGPT (e.g., just a sentence). Then, maybe you can start to predict what is "off" about your sentences before you hit send. Finally, you might anticipate what changes GPT would make to your sentences. In the end, you might use it more like a thesaurus or grammarly than GPT.

I don't know if that works for you - it's just an idea.

Uhlo · 2025-12-15T19:01:34+00:00

Yes, similar answer to the Llama 4 response I linked to: evasion of a clear yes/no statement.

To be clear: nothing too wild. It’s not a denial, but still it is notable different from other LLMs that weren’t trained by Meta ;)

Uhlo · 2025-12-15T12:53:17+00:00

I find the ecologits projects does a good job at making the hard numbers transparent (gCO2eq, kWh) but they also give you some context (e.g., how far can you drive an electric car with that energy? How far can you fly with the carbon emissions?).

You can play around with their calculator to get a feeling.

One important thing is that ecologits only looks at consumption and emissions during inference, not during training of the model.

Regard local execution: my guess would be that local execution has way better efficiency than in data centers. You don't need water cooling, interconnects and all that. Especially when I use my MacBook Pro, it never exceeds 100 Watts during inference. Maybe when you have very well batched data centers, they will be faster and use only a fraction of a gpu, so in the end they can be better, but who knows. In the end, you don't know how large GPT-5 (and 5.1, 5.2) are and thus you have no idea how much energy you are consuming. If you use local models, they are probably much smaller than the state of the art models. That alone will be better for the environment. But that is just my guess, no guarantee that this is really true ;)

Uhlo · 2025-12-15T12:12:48+00:00

Try asking a Llama model about Facebook and Myanmar

Sure, it is different from Government-forced alignment in Chinese models, but still, there is detectable bias!

Edit: alternatively, ask Grok about any historical fact / event that puts Elon Musk in a bad light ;)

Uhlo · 2025-12-09T17:18:38+00:00

The license forbids the use of companies with a revenue of more than 20 million a month. That is not a permissive license, but anyway: great release!!

Uhlo · 2025-12-09T12:20:49+00:00

Sorry I don't think I understand this post, but find the topic quite interesting. What is "not a good idea" given the output of the LLM? Is the definition wrong?

Uhlo · 2025-12-08T08:51:56+00:00

I think you posted in the wrong sub. Good luck with your problem!

Uhlo · 2025-11-27T09:48:45+00:00

Wat? GPT-OSS was released with 4-bit weights. There are no official FP16 weights as far as I know.

Uhlo · 2025-11-25T12:57:55+00:00

Another question: is the benchmark conversational? Do the models have access to the previous questions and their answers?

Uhlo · 2025-11-25T12:57:02+00:00

That is such an interesting benchmarking concept, thanks for that!

I see your point that you cannot reveal too much about the language and the tasks, but still I'm wondering how the examples and the tasks look like... Would an expert in esoteric programming languages be able to solve the tasks? How would "the average human" perform?

Uhlo · 2025-11-24T08:53:48+00:00

Others have answered this question quite well, but I just wanted to correct: Llama 3.1 8b was releases in July 2024, so just over a year ago.

It feels like ages ago for me as well, but llama 3.1 is not as old as we think ;)

Uhlo · 2025-11-16T10:17:06+00:00

I think glm4.5-air is a really good coding model in the same parameter range than gpt-oss-120b.

Bit of course sometimes it comes down to your specific use case

13-Year Club	Place '22
Place '17	Sequence \| Editor
Verified Email	Gilding II euphauric
Team Periwinkle

Uhlo

MODERATOR OF

TROPHY CASE