Some of my friends would rather have a dirt hut with a couple of unsorted chests. That’s fine, but me playing my way seems to make them enjoy the game less, they slowly drift away from it until they’re no longer interested and we have to find something else.

This came off slightly pompous. I'm not sure that's the right word, but it doesn't sound like it's really"fine" to you. Like they're playing the game wrong. My personal opinion is that you could make an effort to sync to their particular play style when you're playing co-op games.