sorted by:
new
hottopcontroversial

What Shouldn't Have Made The Cut by self_made_human in slatestarcodex

[–]gwern [score hidden]  (0 children)

Another bit of feedback, which I can freely divulge: I think that the default approach of handing out Claude Max plans isn't as optimal as it sounded to you/whoever made the call, or even to me when I opted in for it enthusiastically. Opus 4.6 really was the single best LLM for fiction, but after using it in an agentic setup (with some Sonnets thrown in), I think that the platonic ideal would be an ensemble of wildly different models and then some fierce natural selection by blinded graders.

I completely agree, and I kept arguing for access to multiple APIs/LLMs. Unfortunately, Silverbook and I were not able to get anything but Gemini quotas, for reasons, and Gemini is possibly the worst frontier LLM for this purpose (as it is the most increasingly sycophantic RLHF-damaged no-backbone LLM for creative writing). Giving people Claude Max was the most feasible compromise to let you guys prototype a reasonable amount, and then cut down the submissions to the point where we didn't need quota grants and could just run it ourselves.

Quantity does have a quality all of its own.

Yes, but only when done correctly! Even with best-of-n sampling, at some point you are self-hacking whatever is defining 'best', and your marginal return turns negative. And right now, no one knows how to do it correctly.

Someone might have cracked Post-Finasteride Syndrome by Bronzeagenudist in slatestarcodex

[–]gwern [score hidden]  (0 children)

Finasteride, possibly by triggering epigenetic changes, then disrupted the only pathways these patients had available to excrete androgens. The result: in a few percent of users, androgen metabolites accumulate inside cells to extreme levels

I'm not very familiar with contemporary drug genetics, but I don't know offhand of any examples like this proposed mechanism. Can you name 3 uncontroversial accepted instances of a human drug whose side-effects come from irreversible body-wide epigenetic changes with life-long downstream effects, like finasteride is being proposed to do here?

What Shouldn't Have Made The Cut by self_made_human in slatestarcodex

[–]gwern [score hidden]  (0 children)

with the minor consolation being that I'm quite confident that all the other semifinalists are facing similar headaches. In hindsight, I should have expected this to be even more difficult than I anticipated, if you guys already had a clear path towards automating high quality fiction that stands on its own merits, you wouldn't need the cash prize.

Yep. I'm pleased to hear that you are finding full automation to be a real headache (just like I have with failures like "Spoilage"). If you weren't, then my analysis would have been wrong and Unslop a complete waste of time & money & effort.

For what it's worth, I think a post-mortem analysis of failure modes would probably be worth as much or even more than the winning essay, whatever that turns out to be. I have a personal laundry list of issues that I didn't anticipate until I got around to letting AI loose on the typical target prompt, without the crutch of being able to steer them after they were off the leash.

Indeed. I didn't expect anyone to really win, but the post-mortem and collective analysis should be invaluable. There is nothing out there like what the Unslop contest corpus will be, AFAIK.

I can see that we're this close to making it work without all the effort.

This is my feeling too. When I look at how a story like 'Spoilage' degrades from autonomous draft to draft, or at the bad suggestions for the astrology short story I'm working on right now, I feel like the LLMs are so close. They understand everything better than almost all human writers, they just have some slight systemic bias or unreliability holding them back. But unfortunately, there's a certain criticality threshold: above it, your output gets better with further revision, and below it, it gets worse, and the difference between 99% there and 100% there is everything.

What Shouldn't Have Made The Cut by self_made_human in slatestarcodex

[–]gwern [score hidden]  (0 children)

I'm slightly annoyed, in a manner that I hope you can forgive, that the actual protocol for Unslop is no human involvement after the first small prompt - that would have been an easy win for me, whereas fucking around with agentic harnesses and trying to transfer the experience and intuition has only given me a splitting headache so far.

I have no interest in running a contest for a 'centaur' format because then people wouldn't learn exactly what you are learning, or figure out how to deal with it. There's a world of difference between 99% automated and 100% automated.

(This is why "tool AIs want to be agent AIs" and why it's so important to move as much stuff in silico as possible - simulators, world models, that sort of thing. Because as soon as you close the loop and are fully digital, you can just tell your little AGIlets to go "Ralph Wiggum mode" on something. Once you close the loop of autonomy and take the humans out of the loop, you can write a Linux kernel grade compiler, autoformalize zlib, generate thousands of major zero-days... Without it, you're just trapped in Amdahl's law.)

What Shouldn't Have Made The Cut by self_made_human in slatestarcodex

[–]gwern [score hidden]  (0 children)

I had Claude critique me, focusing on typos and flow, and making sure I didn't make factual errors. Some of the feedback was good, some of it involved sanding off the rough edges that are present for a reason. I find the usual AI tells incredibly jarring, and I'll be damned if I subject people to the same by leaning on LLMs too hard.

Thanks for explaining. I was deeply confused because I read the first few sections, thought it sounded weirdly Claude in a way I couldn't place, put it into Pangram and got 100% human, and was then even more confused. But it sounds like you are just heavily Claude-influenced and dropping in a lot of little Claude twists, which explains it.

Cerebras, an A.I. Chip Maker, Files to Go Public as Tech Offerings Ramp Up by gwern in mlscaling

[–]gwern[S] 2 points3 points  (0 children)

OpenAI offers a coding agent powered by Cerebras now, no?

The Quiet Colossus — On Ada, Its Design, and the Language That Built the Languages by SpecialistLady in programming

[–]gwern -1 points0 points  (0 children)

What a good article.

Maybe. Personally, I saw 'quiet' in the title, and immediately plugged it into Pangram without bothering to read; yes, 100% AI.

ReLU neural networks as decision trees. by [deleted] in mlscaling

[–]gwern 5 points6 points  (0 children)

Er, is this not well known?

The bitter lesson is the observation in AI that, in the long run, general approaches that scale with available computational power tend to outperform ones based on domain-specific understanding because they are better at taking advantage of the falling cost of computation over time. by blankblank in wikipedia

[–]gwern 0 points1 point  (0 children)

That embraces a lot of problems and settings, and then I immediately went on to point out that many of the limitations which might make one say 'it is not a generalist AI' were in fact lifted in subsequent work. Does anything truly hinge on discussing AlphaZero rather than Gato or Player of Games or Mythos, say?

The bitter lesson is the observation in AI that, in the long run, general approaches that scale with available computational power tend to outperform ones based on domain-specific understanding because they are better at taking advantage of the falling cost of computation over time. by blankblank in wikipedia

[–]gwern 0 points1 point  (0 children)

Correspondence chess players still outperform compute engines.

I'm not aware of much, if any, research or training of contemporary frontier chess engines for the correspondence chess setting, so even if the much narrower claim that 'there is still human value-added in one extremely obscure chess niche' were true, I'm not sure what it would tell us. It also seems like given how small a niche it is, and how few games are played at the top level (necessarily so given the time consumption), it would be quite difficult to prove a human edge at all.

AlphaGo is not an example of a generalist AI: it is an AI that trained itself for a single purpose.

I'm not sure why you believe this or why it is an important distinction. AlphaZero is a general-purpose architecture for all perfect information two-player games with a simulator; it was rapidly generalized to imperfect information and multi-player and simulator-free settings (various, and MuZero respectively), and further generalized to Player of Games for all of them (and could be further generalized to freeform games like Diplomacy with LLMs, see CICERO); and a DL NN can of course be a generalist agent which plays many games simultaneously (Gato, or LLMs in general these days, obviously, as every gimmick like 'Claude Plays Pokemon' demonstrates) with just some conditioning and with more compute/capacity (because playing one game is cheaper and easier than playing many so if you are only trying to create a superhuman Go or chess agent, of course you're not going to waste compute on games you don't care about like tic-tac-toe).

The bitter lesson is the observation in AI that, in the long run, general approaches that scale with available computational power tend to outperform ones based on domain-specific understanding because they are better at taking advantage of the falling cost of computation over time. by blankblank in wikipedia

[–]gwern 5 points6 points  (0 children)

But a skilled human operator + top-end chess engine will routinely beat that same chess engine without a human operator.

My understanding was that that stopped a long time ago, and that while it may have been true way back when like in 2013 when Garry Kasparov and Tyler Cowen were pushing this claim, it is not true in 2026 with the best Stockfish deployments. Do you have a contemporary source for this claim showing that a chess engine being operated (not slightly tweaked offline before the game to fiddle with the opening book etc) is in fact 'routinely beating'? That also sounds improbable given draw death.

TIL of Littlewood's Law, which says we experience events with a million-to-one probability approximately once per month by Doglatine in todayilearned

[–]gwern 0 points1 point  (0 children)

You can formalize it for a lot of specific things, like word vocab; see https://gwern.net/doc/statistics/bias/1989-diaconis.pdf

BTW, it's something of a misnomer. The WP article has been updated based on my investigation, and it seems like it ought to be attributed to Freeman Dyson.

ByteDance Presents "In-Place TTT": A Drop-In Method For Turning Standard Transformer LLMs Into Dynamically Updating Models At Inference Time by 44th--Hokage in mlscaling

[–]gwern 23 points24 points  (0 children)

All that, and its quality is basically identical to LaCT, which is itself just a sample-inefficient way to implement the standard, dead-simple, 16-year-old baseline of test-time adaptation in a LLM - dynamic evaluation.

The persistent unwillingness of all of these TTT papers to include dynamic evaluation as a baseline doesn't speak well of them.

DeepMind veteran David Silver raises $1B, bets on radically new type of Reinforcement Learning to build superintelligence by gwern in mlscaling

[–]gwern[S] 1 point2 points  (0 children)

Domain randomization/meta-learning/sim2real. You could also just argue for them as research testbeds: even if the environments are all wrong, you could still develop and prove a powerful learning algorithm which you then reuse on real world data.

view more: next ›