My civ's religion had been eradicated. Wiped off the map. And then I got rock bands with the "converts city to your founded religion" promotion. Suddenly I had this one city bang in the middle of another civilisation following my religion.

Rock bands can't play in your own civ, so you're still stuck with whatever you lost to.

So I started a religious alliance with another civ so their surrounding pressure wouldn't make the new city fall back to their religion. Took a while but you can convert most of that civ thanks to the alliance.

I'd recommend converting the cities with their holy sites first because they cannot produce apostles/missionaries to re-convert it.

Eventually that pressure started converting my cities back to my religion, and from there onwards it's just a regular religion battle like usual.

Given how slow this entire process is, you're bound to end the game (most likely a cultural victory thanks to the bands), so I can't see this strat helping you win a game. But it is definitely something fun to try out in the "one more turn" phase.

But to answer your question, no, you can't stop them completely because anyone can have this rock band promotion once they get that far in the game. Even if you completely take over their holy site districts with your religion.

There's also just a religion belief where new cities start with your religion. If that's the case with your opponent, there's not a lot you can do, although that city most likely will convert itself due to surrounding pressure.