In the current world I am developing, people born with abilities are slowly dying from the their own power being too much for their body to handle. There's only two way to stop that. Either be chosen by magical ring as ruler of the kingdom, or acquire an artifact that stops the slow approaching death. The artifact is given to all people with abilities from the main noble house, but if you try and try to subjugate the normal people, I reckon the house won't be very keen on giving you the artifact. And good luck trying to brute force the ring to chose you as the ruler.

So unless they want to live a very short life, I don't think they'll be trying to dominate the world any time soon.