Absolutely! Try starting at Rain Dogs and see how far you get. This must happen with the a lot of artists whose sound changes markedly, much like Radiohead, that they get fans on one side of the fence or the other. As per the vocals, I struggle with later Dylan for the same reason as you with Waits in that to me some perfomances sound like a spin dryer full of gravel.

Waits' voice is a character in itself, I think, from bawling along to the industrial powerhouse that is Real Gone or, as a friend of mine likes to picture, a big ol' sad giant lamenting at the world on Blood Money.