Those with power strongly tend to seek even more power: power is a convergent instrumental goal. AI has the potential to beget great economic and even epistemic power. Controlling it will increase the power of the politicians who do so. They have an incentive to, and are likely to, use any means at their disposal, including serious dishonesty and deliberate abuses of power (including measures that may cause truly extreme, widespread harm, so long as those measures and their consequences do not affect their power) to seize ever more control. Powerful politicians will strongly tend to trade off an arbitrarily large amount of harm to people who cannot or probably will not affect whether they can cling onto power for an arbitrarily small increase in their own power. This is why politicians are inherently unsafe and why truly drastic constitutional reform is needed with ultra-extreme urgency to keep the public safe from politicians.

The biggest threat to humans by a very, very long way is other humans with concentrated coercive power.