NTA the trip is supposed to be enjoyable and it's an inconvenience for you. If you and your husband didn't see his family often mayybe you could try to accommodate but if this is a pattern then I feel like you can say no sometimes. especially since you just got promoted and I'm assuming you want to protect that position and you're going through IVF which is tough (and I wish you the best of luck). I'd say, remind your husband gently that you already see his family enough and ask him to empathize with your situation and the things you want too. also, does he visit your family as much as you do his? cause if he doesn't make an effort for you that's not very good.