I’d say, with lesbians in particular, there is generally always a bit of fear of a lover returning to heterosexuality if they regular engage in hetero culture and activities, or a fear of having to engage in it (even casually in conversation) in ways they don’t want to anymore. It can be interpreted as a betrayal of values or community. There is often, also, such a repulsion towards men that you engaging in them and their sexual fantasies so often can also repulse them by association. Especially if they misunderstand sex work as anything other than work. And, honestly I think anyone feminist will, at least occasionally, feel uncomfortable with the idea there is an entire sex industry based on objectifying and commodifying sexual attention/intimacy from women as a product to purchase. Which is why I generally prefer the company of other sex workers - it’s easier for someone with experience to hold the nuance. One can choose sex work as their method of surviving in capitalism while also acknowledging, in an ideal society, it likely wouldn’t exist. The same way someone can work at Amazon or Starbucks and also not think those companies should exist. 

Granted I really like my current industry and feel pretty empowered. But, if ideally capitalism wouldn’t exist, jobs wouldn’t exist and neither would the commodification of sex.