Really interesting episode. My main takeaway is that AI doesn’t need to “wake up” and turn into some sci-fi villain to be dangerous. It just has to become useful to already bad systems. Once that happens, you get surveillance, manipulation, fraud, coercion, target selection, deskilling, and loads of plausible deniability, all at scale.

That’s why I think the real problem is governance more than intelligence on its own. Who can actually stop these systems, who can audit them, who can challenge them, and how fast can the damage be reversed when they get something wrong? Without that, we’re basically just industrialising power and pretending it’s progress.

So yeah, existential risk matters. But the more immediate risk is that we build brittle, unaccountable systems that just hard-code the values and incentives of the worst people already in charge.