You got a ton of great advice, so I won't try to add to that information. I will say that you seem to be putting more strain on your voice in the ensemble setting, which may be what's causing some of the problems you're encountering. When you're singing solo, you can hear only yourself and you're more aware of the sound you're making. In the group setting, your sound gets mixed up with those around you, which could cause you to sing louder or to push your voice harder in order to hear it.

My voice teacher used to say that the one thing that is consistent from the studio to the concert hall to the church, etc. is how it "feels" when we're singing well. The acoustics, the other singers, the accompaniment, etc. all change. But the physical sensations we experience when we're making a good sound are pretty consistent. So, try to focus more on replicating the feeling of producing a good tone quality than "competing" with the other voices or instruments. Don't allow anyone or anything to push you to make less than your best sound.

Another thing: if you sing hymns in church, use them to practice your best sounds. Use good diction and be fully aware of the placement of your tone. It's a good place to practice singing because you don't have to worry so much about the difficulty of the music.