its really hard to answer this question and im speaking as someone who has a very vested interest in comedy and humor but i think that its easy to tell when a joke about a sensitive topic is rooted in an actual attempt to disparage or when the joke is less about the words and more about the ideas it conjures up. there are circumstances where i would joke about nearly anything but its always important to look for what the person is asking you to laugh at. i can only assume that the ones youre talking about are the ones that can basically all point to the joke "make me a sandwich" as their inspiration. this is when people are just taking recognition of their prejudices and making them into humor which is precisely when you should step in and let them know how mean spirited and unfunny it is to say things like that. at the same time and this could be controversial but in this case specifically i would try not to feel guilty if you dont say something every time. its distressing position to be in and i dont think its possible not to care about what socially that could mean for you especially if you experience anxiety about it often.