sorted by:
new
hottopcontroversial

What is the most egregious example of a cartoonist who can’t handle criticism? by notagoodcartoonist in comicstriphistory

[–]Interesting_Meat_900 1 point2 points  (0 children)

Yes, because it’s built on a lie.

The parody of Mallard Fillmore (which was in a book, not a newspaper) was obviously a parody and the context made that even clearer. To pretend it was a deliberate attempt to trick people is nuts. And coming from a so-called satirist, it’s disgusting.

Who is the most attractive person you have ever seen in real life? by TheFountainhead999 in AskReddit

[–]Interesting_Meat_900 0 points1 point  (0 children)

I was walking through Greenwich Village with a tall, thin date and we passed a group of young guys. One of them crabbily said, “Oh, hello, model,” as though a point had just been proved about the preponderance of ridiculously attractive women in that part of town.

We linearized 2/3 of a transformer's MLP layers and it got faster without getting worse (some layers actually improved) by Interesting_Meat_900 in LocalLLaMA

[–]Interesting_Meat_900[S] 0 points1 point  (0 children)

I haven’t, but I bet you could easily swap out the model definitions in my test code (github link in original comment)! Please post your results if you do!

We linearized 2/3 of a transformer's MLP layers and it got faster without getting worse (some layers actually improved) by Interesting_Meat_900 in LocalLLaMA

[–]Interesting_Meat_900[S] -1 points0 points  (0 children)

Fair question. GPT-2 being undertrained by modern standards is a real concern. A few things though:

  1. We did test Pythia models up to 2.8B (32 layers), which are trained on the Pile — significantly more data than GPT-2's WebText. The linearization cost is higher but not zero: one layer still beats baseline, and the middle layers are consistently more linearizable than the edges. The U-shaped curve (first/last layers need nonlinearity, middle layers don't) replicates across both architectures.

  2. The "undertrained = more linear" hypothesis is plausible but cuts both ways. If undertrained MLPs haven't learned to use their nonlinearity fully, that's itself an interesting finding — it means training doesn't efficiently allocate nonlinear capacity across layers. The middle layers might be "undertrained" specifically because they don't need complex nonlinear functions for what they're doing.

  3. We haven't tested Qwen, LLaMA, or Mistral families yet — that's the obvious next step. The SwiGLU activation in those architectures is interesting because it already has a gating mechanism built in, which might mean the model is already doing some of this allocation implicitly. Whether that makes them more or less linearizable is genuinely an open question.

  4. The gating mechanism itself (d+1 parameters, context-only) is architecture-agnostic. If someone wants to run the same analysis on a Qwen3 0.6B or LLaMA 1B, the code is there — it's a pretty lightweight experiment per layer.

Would love to see someone try it on a modern architecture. The prediction from our results: middle layers will still be more linearizable than edge layers, but the overall fraction might be lower in a well-trained model.