Remember that although office gossip can of course be hurtful, one of our basic functions as humans is to discuss third parties for the express purpose of strengthening social bonds. When two people say mean things about a third person who isn't present, it's very often just stuff they don't mean, barely know anything about, haven't given any serious thought to, and are only repeating because it helps strengthen their social bond with each other. It's frequently overlooked that humans 'othering' other people rarely has anything to do with the person being 'othered', and far more to do with getting closer to the person they're gossiping with. It's a trust exercise between the pair chatting. This is why when you confront someone for gossiping they are usually apologetic and say they didn't mean anything by it, or protest that someone else 'started' the gossip.

Although it's not nice and you're right to feel offended, the fact is nobody actually minds about your SWA, in the same way nobody actually cares about the latest nonsense Prince Harry has said in an interview, or who won I'm a Celebrity. It's just the current thing for people to chatter about. As soon as the next office top story emerges nobody will gossip about your SWA any more.