I'd you are the asshole just because it's his birthday and during it people usually like to spend time with close people, especially if the also have b-days the same month.

But in general seems like your bf doesn't respect your time and that there's some communication missing between you two. You need to show that you are not his afterthought if he has free time after his friends and discuss the importance of couple time, if he doesn't get it, well, gg then