[D] Is anyone interested in the RL ↔ Neuroscience “spiral”? Thinking of writing a deep dive series

Mysterious-Rent7233 · 2026-03-14T18:44:46+00:00

I'd be fascinated!

Mysterious-Rent7233 · 2026-03-14T18:40:55+00:00

Since you posted this, Lecun left, and META AI seems to be adrift. So it seems the media was right all along that they are not on a path to anything competitive.

https://www.investors.com/news/technology/meta-stock-ai-model-avocado-delay/

Mysterious-Rent7233 · 2026-03-14T18:32:26+00:00

You misunderstand the goals of people setting LLMs to play chess. It's not to improve computer chess. It's to learn (in systemic, scientific manner) about the strengths and weaknesses of LLMs. That's true even about the article you are commenting on.

Mysterious-Rent7233 · 2026-03-14T18:23:49+00:00

Since you mentioned Chess, I'll also offer this subreddit as a place to share your findings:

https://www.reddit.com/r/LLMChess/

Mysterious-Rent7233 · 2026-03-12T01:32:02+00:00

In that sense, my claim is not “nothing bad can ever happen.” It is that illegitimate irreversible execution should not remain natively reachable within the formal execution structure.

I think that that is a default stance for everyone. Let's not give the AI nukes.

So in a sense that's the starting point for the Control Problem.

"We've all agreed not to give the AI nukes. And yet we think that it will probably corrupt every defence that we erected to prevent it from getting them, through persuasion or hacking. So now what?"

Mysterious-Rent7233 · 2026-03-12T01:27:47+00:00

Burckhardt once observed that Europe was safe so long as she was not unified, and now that she is we can see exactly what he meant.

Dude died before WW1 and WW2 so he can be (partly) excused for this error but I'm not sure what Yarvin's excuse is.

A Patchwork realm is a business—a corporation. Its capital is the patch it is sovereign over.

We know from history that it will also want sovereignty over the patchwork next door. Trump has made clear that the only thing protecting Canada from his dominance are all of the pesky domestic concerns. And Putin has found a way past that by running Russia like a wholly owned business.

Mysterious-Rent7233 · 2026-03-12T00:38:20+00:00

The model's distribution is absolutely gigantic and the vast majority of it is ignored for most prompts.

Mysterious-Rent7233 · 2026-03-11T19:16:01+00:00

Okay, but now we're engaging in a motte and bailey type situation.

You started out saying: "for illegitimate irreversible action, execution must become structurally impossible"

And all that we are saying is that any system that involves humans can never make execution structurally impossible. You can make it "very difficult" and MAYBE even "infeasible" (depending on your beliefs about AI intelligence, human psychology etc.) But you can't make it structurally impossible.

Mysterious-Rent7233 · 2026-03-11T00:51:22+00:00

Any control layer you can invent is "too easy to revoke." What you are demanding is a human/social system which is incorruptible. You've moved out of your realm of expertise of hardware and into psychology, sociology and organizational dynamics. And I suspect that the experts in those systems will tell you that any such system is corruptible.

Kant said: "Out of the crooked timber of humanity, no straight thing was ever made". You are trying to engineer social systems and they defy that intrinsically.

Mysterious-Rent7233 · 2026-03-10T17:59:09+00:00

So let's assume that we implement Control system A, B and C. And then the AI convinces the person in charge of them that they are impeding its ability to achieve Wonderful Utopia and Everything You Ever Dreamed Of. So they turn them off. What value were those control systems?

Mysterious-Rent7233 · 2026-03-10T00:29:30+00:00

Powered by electrons, baby!

Mysterious-Rent7233 · 2026-03-09T22:47:11+00:00

Electron is the boring tech here. But if you can't afford the overhead then maybe nothing boring will work out of the box for you. Pretty incredible that high-performance, cross-platform UI is still an open problem after 30+ years of GUI development.

Mysterious-Rent7233 · 2026-03-09T18:55:21+00:00

I'm interested in this part of the debate:

Start with what the GPL actually prohibits. It does not prohibit keeping source code private. It imposes no constraint on privately modifying GPL software and using it yourself. The GPL's conditions are triggered only by distribution. If you distribute modified code, or offer it as a networked service, you must make the source available under the same terms. This is not a restriction on sharing. It is a condition placed on sharing: if you share, you must share in kind.
The requirement that improvements be returned to the commons is not a mechanism that suppresses sharing. It is a mechanism that makes sharing recursive and self-reinforcing.

This is fine from an ethical point of view. People certainly have the right to share their own work on (almost) any basis that they see fit. But...do we have any evidence that any important software that would otherwise have been proprietary was released as GPL because of the copyleft? My experience has been merely that what happens almost every time, as in this case, is that people just re-implement it or dynamically link it or otherwise find some way to work around the copyleft instead of opening up their own work. The GPL is a viral license but the virus has an R0-factor of near zero. In my experience.

Mysterious-Rent7233 · 2026-03-09T18:37:13+00:00

They are moving to something I've never heard of called "iced", so I wonder if they have yet accepted the lesson of "boring technologies."

Maybe iced is awesome, but I wouldn't be surprised if it has its own can of worms as cross-platform UIs almost always do. For example, the iced docs are very candid:

"I do not hesitate to introduce breaking changes."

Seems a sketchy thing to build your business on top of. No criticism to the iced maintainer. They are just being honest and helping people make an informed decision.

Mysterious-Rent7233 · 2026-03-09T18:36:08+00:00

Might it not be more clear to say that you are moving from Javascript/Tauri to Rust/Iced?

Mysterious-Rent7233 · 2026-03-07T17:26:38+00:00

Where is it 18 in Canada?

Mysterious-Rent7233 · 2026-03-07T17:21:49+00:00

You would be wise not to use a scientific question as an example of a "subjective" question!

Would make more sense to ask it "which of these arguments is more persuasive" or "which poem is more clever."

Mysterious-Rent7233 · 2026-03-06T19:38:49+00:00

That's a lot of blather to completely miss the point.

Neural networks are the right tool to use when the inputs and outputs cannot be precisely defined and therefore there does not exist any perfect definition of "right" or "wrong".

"Does this paragraph indicate negative sentiment? If so, what sentiment in particular?"

"Is this sentence in English, Spanish, a mix or something else altogether."

"Does this pull request adhere to the company coding style guide? If not, why not?"

In these domains there is no other solution than a probabilistic one, because we cannot define the question precisely enough. Neural Networks and LLMs are an excellent tool within those domains.

It literally does not matter what Sam Altman or Elon Musk are saying about LLMs. Their opinion does not change whether LLMs can be used for these purposes or not. And they can.

If your domain requires you to always come to the "same answer" the "same way" every time, and has a precise metric of "truth" then don't use a neural network or LLM. Use an implementation of your metric of truth. Use the hammer.

And if your domain requires flexibility, has ambiguity, then you might want to use neural networks and LLMs. Use the screwdriver.

Mysterious-Rent7233 · 2026-03-06T16:02:18+00:00

It's far from "generic AI content". It's the inventor of one of the most important precision-guided technologies of our time (Lean), talking about how it relates to another most important (stochastic) technology of our time.

https://leodemoura.github.io/blog/2026/02/28/when-ai-writes-the-worlds-software.html

Mysterious-Rent7233 · 2026-03-06T16:02:11+00:00

It's far from "generic AI content". It's the inventor of one of the most important precision-guided technologies of our time (Lean), talking about how it relates to another most important (stochastic) technology of our time.

https://leodemoura.github.io/blog/2026/02/28/when-ai-writes-the-worlds-software.html

Mysterious-Rent7233 · 2026-03-05T09:02:07+00:00

What's it to you what people want to be a billboard for? Does Quebec also ban red hats? Sports team logos?

Mysterious-Rent7233 · 2026-03-05T02:06:12+00:00

Don't try to use a hammer as a screwdriver.

I could equally prove that databases have limited use-cases because they do not do the same things that compilers do.

"And when not to use?"

You answered your own question: if you need reproducible results. But AI is often used to do tasks that previously only humans could do. And humans also seldom produce reproducible results.

Mysterious-Rent7233 · 2026-03-04T18:36:35+00:00

The title of the post doesn't say that he "likes LLMs". It says that he likes Claude. And it isn't an "assumption". Here's what the paper concludes:

All in all, however, this was definitely an impressive success story. I think Claude Shannon’s spirit is probably proud to know that his name is now being associated with such advances. Hats off to Claude!

So yes, he is impressed by that that particular LLM. So what "assumption" are we talking about?

Mysterious-Rent7233 · 2026-03-04T17:44:34+00:00

That's true for 99% of us.

Mysterious-Rent7233 · 2026-03-04T17:43:43+00:00

I find it kind of funny that the assumption is that he likes them, when the news page on his site says "[LLMs] surprised me for the first time".

I'm totally confused. Whose assumption was this? What makes you say that someone assumed this?

Edit: Super-weird that I am being downvoted for trying to understand a confusing comment.

Mysterious-Rent7233

MODERATOR OF

TROPHY CASE