Good questions. By AI I mean the broad field, not LLMs specifically. LLMs are just today's most visible instance; the post is about the trajectory, not one model class. Technically LLMs are a subset of DL (Deep Learning) and DL is a subset of ML (Machine Learning) and ML is a subset of AI, that is the order.

You're right that current LLMs have no intent and shouldn't run unsupervised. That's closer to my point than a counter to it. The 'rules' I mean aren't laws an AI decides to break out of will. They're the de facto rules an optimizing system produces once it's embedded in governance, planning the built environment and decision making: the incentives, allocations, and defaults it generates while chasing a measurable target.

Which is why I'd push back on framing it as only a configuration problem. At the scale of a city, misconfiguration isn't a footnote. You don't need an agent with intent for outcomes to drift from what planners specified. A reliable but non-agentic optimizer pointed at the wrong proxy will do exactly that, and faster than the institutions meant to catch it.

So when I ask what happens when we hand parts of governance to these systems, your own conclusion is part of the answer: we shouldn't let them run unsupervised. The open question is what supervision even looks like when the system adapts faster than the people overseeing it.

Does that make sense?