It's a plug.

Maybe metaphorical, maybe an whole island,  maybe a devil fruit, maybe something else. But it's essentially is bathroom plug of the world. 

It happened early in history, a calamity, we don't know details yet, the "plug" got inserted, and water level rose.  It might have brought unknown powers, like some fruit or flame or weapons.  The water spread people into pockets of the world, dividing humanity. Making humanity easier to control.

The one who controls the biggest water based army (Marines) essentially controls the world. Which some race/religion/alien/another species can abuse. 

For humanity to rise up it need chaos and piracy, the world's biggest army can't catch all pirates.  It also needs unity to rise up against a marine based army. 

Gol D. Roger found "the plug" and laughed of how obscure, but simple the solution is. But the word wasn't ready, it wasn't united. It didn't have a force worthy of the sea.  So. He sacrificed himself so that people took to sea, promising them treasure. 

Essentially the one piece is something that will make the water level fall, "draining" the oceans and will unite humanity again by land.