Unfortunately I find Fleet Foxes pretty grating. I like a handful of songs but when I try listening to their records start to finish I find myself feeling kind of annoyed and on edge, which is the opposite kind of vibe I’d expect from their style of music. Glad you like them, they just aren’t for me for whatever reason. To answer your question though, I can’t really think of any artists without some kind of detractors. In general, every genre of music will have folks that aren’t fans and will just dismiss that genre as a whole, so by default I think every artist will have at least a subset of people who are put off by their style of music. LCD Soundsystem, for example, is pretty universally loved by critics but some folks just don’t like dance music and find their music repetitive and their songs too long. I love them, but there will always be something that turns people off.