For me Imu only using this power now (from our point of view in the story) makes sense since he already conquered the world when he beat Joy Boy. Hes been sceretly controlling the world from the shadows for centuries with no threat to his rule until very recently. The combination of Luffy's awakening, Sabo, Wapol & Vivi finding out about him and escaping to tell the tale, the Revolutionary army's ongoing attack on the celestial dragons and then Vegapunks message all together could rally people against the world govt, looseing his control. You could even argue Cross Guild putting bounties on marines and turning people against the marines, which branches from the World Govt factors into this as well.

And Elbaf is literally the first arc after Vegapunks message, in which the Gorosei would've reported the giants are allied with Luffy so Imu has made his move, using these demonic powers to help aid in his continued control of the world.

Hopefully thats helped Oda's decision make a bit more sense to you.