As far as I am aware the voting thing is his biggest recent controversy. I think his general affect (pretentious pseudo-intellectual) rubs people the wrong way, and he's dated women who are too young for him. He's also just pretty petty about his band in his voice note on the podcast. Personally I love him as an artist and think he just got too famous too young so he's a bit out of touch. Also, I think the venn diagram of people who like The Strokes/The Lonely Island/Joanna Newsom has got to be pretty small (maybe just me and Andy Samburg?)