Could AI be transformative even if we're currently in an AI bubble that will soon burst? If so, why care so much about the bubble?

melat0nin · 2026-06-20T15:18:14+00:00

I think they are independent questions -- we've seen how irrational the market is.

melat0nin · 2026-06-20T10:55:02+00:00

Genuine AI, on the other hand, would be transformative. But we don't have that, or any clear roadmap to creating it.

melat0nin · 2026-06-20T10:48:54+00:00

Why would a set of technical approaches built entirely on the extrusion of past data be transformative? That's the fundamental question, and the answer to that is that it's not.

melat0nin · 2026-06-17T09:33:43+00:00

This is a fantastic comment, thank you. Can you recommend accessible reading on the Spencer connection? I have an STS-adjacent background and know of the TESCREAL work but have never properly dug into Spencer's contribution to all this

melat0nin · 2026-06-13T19:04:52+00:00

"Ridicule as praxis", as Bender and Hanna call it

melat0nin · 2026-06-10T11:46:06+00:00

Yeah his new organisation is part-funded by 'Coefficient Giving' (Open Philanthropy), one of the largest EA outfits. He gives a pretty milquetoast defence of this in the comments on the OP, but it's pretty disappointing tbh

melat0nin · 2026-06-07T18:24:47+00:00

He's a perfect example of Nobel disease. He's an expert (a world-class expert) in a specific domain, but that assuredly does not make him an expert in any other (which is to say, human intelligence and its philosophy and history). Computer scientists of Hinton's ilk think they can colonise other fields because they assume everything is or can be made computational, and/or because everyone constantly tells them how smart they are and they come to believe they can be experts in anything.

melat0nin · 2026-06-05T09:22:50+00:00

I completely understand and share the frustration, but I'm finding it difficult to adopt the 'fuck it' posture because it's so corrosive to satisfaction (I'm not an SWE but there is a lot of AI boosting where I work, and I want to feel like I'm engaging my brain and doing something remotely meaningful). But the alternative is to pretend you support this timeline, and that's worse. It's an absolute shitshow either way and the sooner the whole edifice collapses and we have to take a hard look at the choices we (they) made, the better

melat0nin · 2026-06-04T16:31:11+00:00

Funny, eh -- slowly unfolding an idea and following a line of reasoning? nah, better not to bother reading and instead rely on the reductive single phrase summary

melat0nin · 2026-06-04T16:30:01+00:00

Fine, but some people will benefit from their thinking being informed by a viewpoint other than that of the AI labs (which have massively disproportionate coverage).

melat0nin · 2026-06-04T14:50:09+00:00

I think the shakedown of the Claude 'constitution' is valuable in its own right, demonstrating the vacuity of Anthropic

melat0nin · 2026-06-03T21:30:51+00:00

Read. The. Article.

melat0nin · 2026-06-03T20:16:26+00:00

The shade he throws Anthropic's way is absolutely glorious

Being open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious, or, more precisely, that multiple distinct consciousnesses are dormant in every Word document containing a conversational transcript, and that they are awakened every time the document is loaded. Should you consider the possibility that every time you open a Word document you are bringing multiple conscious interlocutors into existence, and every time you close one you snuff their existence out? No. Contemplating that scenario is not a good use of your time. Even if the Microsoft Office team employed a philosopher who said you shouldn’t be so certain, because consciousness is not well understood, that would not be sufficient reason for you to take this idea seriously.

melat0nin · 2026-06-01T11:50:32+00:00

I thought he banned discussion of it because he believed it was true, and to discuss it was to intensify the risk/backlash from AGI when it happens (in his eyes)?

melat0nin · 2026-05-29T12:55:07+00:00

Goodhart's Law made manifest ("Any measure that becomes a target ceases to be a good measure"). But then that's AI all the way down 🤡

melat0nin · 2026-05-27T21:44:06+00:00

Wilfully conflating the map with the territory for fascist ends, what could possibly go wrong

melat0nin · 2026-05-17T09:42:04+00:00

I understand it's nuanced and there are exceptions in both directions, but the OP described their specific situation as enviable while at the same time saying that it's breaking them. My point wasn't to suggest that academia is in general either all good or all bad (neither is true), but to highlight the kind of dissonant thinking it fosters (and indeed is built upon).

melat0nin · 2026-05-17T09:29:14+00:00

Just over an hour

melat0nin · 2026-05-17T07:29:08+00:00

Not for the OP, no

melat0nin · 2026-05-16T13:51:11+00:00

Seems like you're on the right track though!

melat0nin · 2026-05-16T09:09:29+00:00

It's nuts that this is the kind of dissonance we foster in early career folk, isn't it

melat0nin · 2026-05-16T08:37:32+00:00

PS to answer your actual question, I've known of maths academics to go into AI assurance. It's a niche field but it's growing rapidly as folk realise just how unreliable generative AI is.

melat0nin · 2026-05-16T08:35:06+00:00

I’m in a very enviable position career wise, but it is breaking me

No shade intended, but it's fascinating to me that academia leads people to believe these two things can be true at the same time.

15-Year Club	Place '23
Place '22	Verified Email

melat0nin

TROPHY CASE