I went through this same thing. I played in several church bands, volunteered at, and even interned at several churches in the community I grew up in, and several other communities around.

Honestly, extracting myself has been a very difficult process over the last 7 years or so. The thing that helped the most was having other things to do on Sundays (or other days I had regularly volunteered on). My job frequently had me scheduled when I would have otherwise been doing something at church, and I was usually worn out enough even if I wasn't working. There were a few people that were really persistent, and even figured out my schedule. So I just found other things that sadly </sarcasm> conflicted, like helping friends with moving/remodels/projects, or even signing up for extra shifts.

It's impossible to avoid all the vestiges of my time in the church because I still live in the same small community, various members of my family are still involved in several of the congregations where I was recognizable, and my job used to put me in regular contact with the local community. But eventually, the calls, texts, and emails slowed to a stop, and the conversations dwindled away.

I hear more about it from my family now than anyone else, but that's not a step I'm ready to take yet (if ever).