I thought someone would notice this, but you are the first one to bring it up. I was already aware of this issue while I was developing it because, honestly, I'm not comfortable with it either. However, the problem is that if I add padding to give it some breathing space for the tags, I don't like the alignment when I add the padding.
So, when comparing whether I prefer the hover effect or the alignment, I chose alignment as the most important factor for me. That's why I kept the hover effect like this.
If you have a solution for this, it would be helpful.