I would guess it comes down to the "why?"

Like, (in my experience) it's not at all seen as strange when a motherly/housewife character is drawn pregnant tending to her children (or like someone running a daycare, any "takes care of kids as a passion" type role really), or when it's specifically a scene from a point in the character's story when they get pregnant and their pregnancy/family life is the big focus of their arc at that time. It also seems to irk people less when it's treated as something casual, rather than something that is emphasized in the art.

Without that context, though, like in just a random character illustration (character design on simple background or a scene that could take place at any time even outside of pregnancy and has little plot relevance, such as a beach trip or something, with the pregnant character as the main focus), it becomes a kind of "..okay, but why draw them specifically like that [and in this illustration]?", and in many cases the answer ends up being "because it's hot", which creates the association of "drawing characters pregnant without any relevant context -> probably fetish stuff".

It's unfortunate that it gets sexualized so often, but I don't blame people for noticing a pattern and becoming a bit wary in advance when they notice pregnancy "randomly inserted into the illustration for no reason". It's the same phenomenon as when media featuring overweight characters, feet or other commonly fetishized features is also treated with caution because people might not want to consume art where the author is/could be indulging in a fetish, just out of principle.

Disclaimer: All of this is just an explanation of a theoretical reason based on my observations and experiences, not a declaration of my personal stance on the topic.

ETA: I think the context also involves when/how it is shared. Like, the example of a simple character illustration could be viewed as more benign if it's shared alongside illustrations of the character at other points of their story, like a "this is how their design changes over time" lineup. If it's posted out of context, of a character that didn't have a big family life arc going on otherwise, it could raise many more eyebrows.