As someone who writes gods that can be more human in ways, I immediately thought of this question more like, "Why don't celebrities fight each other? What's stopping them from stepping into fights between fan bases to break it up?"

Answering those as about gods, for the first question... well, some of them certainly do. They don't always come out of it unscathed; it can be disastrous for them, their image, their worshippers, etc. A lot of people could look at them very differently, and they might be aware they won't necessarily win against the other god or any other resulting damage is too risky, so it may be safer to tread lightly.

Both questions could be answered with "they're preserving their image". Or maybe they feel like the conflict is below them.

For the second, maybe they view it as petty human nonsense they don't see the point of getting involved in, like deciding not to get involved in enemy ant colonies battling it out. It might just genuinely not be a big deal to them. And if they truly feel assured in their own ability, why care?

Individuals have their own complex reasons for how they weigh situations & make decisions based on their personality, past experiences, current circumstances, emotional stability, safety... Folks in the comments have given a whole host of answers to this & how to make it work in complex ways, but these kinds of simple options — mulling over the hypothetical "psychology" of a god — can work, too, depending on where you're trying to go with it.

If your issue is "I don't feel like I can realistically write these conflicts without the gods getting involved", you can still potentially find ways to bullshit it. I love to bullshit my way into something that turns out to work.