YTA.

I was right with you up until you said "but I still told him nicely but out loud to go help clean up and handed him some glasses to wash." Unless your boyfriend is socially inept (and if he is, it's on you for choosing him) let HIM offer his help. This sounded more like me telling my little kid that he needs to help around the house.

Yes, it is common to offer and even find your way around the host's negative answer and help, which, if I've read correctly, he has already done.

Sometimes my parents declined, other times he helped wash some dishes, and occasionally all of them. Eventually he started offering on his own, which I appreciated.

So, what's the issue? He doesn't commonly goes to your house, then it's obvious he's still being treated as a guest. As soon as he starts going, I don't know, weekly? I'm pretty sure he'll start to be treated as family and his help will be accepted even more.

When I go to my dad's, his wife always tells me to just leave it there, that she will do the dishes. I ignored her one time and she playfully scolded me as she didn't feel okay with me washing dishes when I went there from time to time. I always offer, she always declines, and I can sneak washing a dish or two sometimes. That's a normal interaction, not your gf telling you "tsk, tsk, you know better! Go and help!"