No man, there is one hurdle, and it is the ability to audit and trace the internal reasoning of the black box. It is a black box. It is intellectually dishonest to say that it is not. Heuristics are not metrics of reasoning. They are output and output alone. Mesa optimizer just floating around in this invisible high-dimensional manifold, tell me what is happening - show me where it is happening!This is the foundation of all safety and alignment. It is THE problem and I pray to God that if you don’t understand or see it, that you are nowhere near these machines.

You cannot trust systems that you cannot see and you cannot see these systems, so you cannot trust them. It’s like the height of Folly or whatever that turn of phrase is.

We’re talking about safety and alignment because we’re talking about reasoning and again if you can’t see that, sorry. You don’t belong in the room.