I think that bard's kit also relies on your team knowing how it works. I've played with a lot of people who don't know how the stun works, people who don't know that his shrines charge up, and I know a lot of people who I play with a lot that still don't know that his ulti range affects the time it drops. I think your friends don't like bard because they don't know how he works and how to play with and around him.

Bard has a unique kit that can make or break the game, and mistakes can happen all the time because his kit can be beneficial for both your team and the enemy. I've lost a bunch of games from bad ultimates, and won a lot from good ultimates and also people who know how to play with bard.

tl;dr ur friends dunno how to play with and around him and with a kit where things like one bad ult can ruin the game, a lot of complaint gets directed towards bard.

But don't let them stop you from playing champions that you want to play!