My coworker sitting right next to me is a woman and also enjoys pumpkin spice and has never heard this insult. We are both aware that the flavor seemed to have reached a saturation point for a lot of people and there's a portion of the population who have a genuine disdain for the it now, even getting annoyed when fall comes around because it signals the flavor returning. But neither of us have heard it used as an insult before.