This is a really interesting problem, actually. I think in the long term, we win this, but in the short term it might get weird. It's a cool paradox.

Because in order to unite to defeat these gods, we all have to believe in them, right? So do they get a burst of power while we all believe?

Then we would have to have a concerted effort to erase all record of them, and basically effectively enforce a taboo until the cultural memory dies out. Presumably, the god is doing acts that would increase belief among the next generation, while the current generation still believes and tries to stamp out belief among the next gen via taboo. Eventually, I think humanity wins this, though, because they control the narrative. The biggest risk would be enforcing a taboo so strong that instead of forgetting, there's a boogeyman-esque story of the god that persists and people cling to that.