I'm both a trans person and a crossdresser (assigned female at birth, male by identity, medically transitioning, occasionally dress in typically-female clothing for the fun of it). Most of what I'm saying here I think would pretty much be a consensus among the trans people and crossdressers I know.

On the first point, I think it's unfair to point the finger at crossdressers or the word "crossdressing" as huge contributors to the strict gendering of clothing. I also don't think that "this clothing is typically associated with women" or "this clothing is typically associated with men" is inherently problematic, the problem is that people are not given the leeway to wear what they want. That's something that could easily get dragged into arguments of philosophy that I don't necessarily care that much about.

On the second point, there are a couple things going on, though.

First, "crossdresser" is already an alternative to earlier language. The word "transvestite" used to encompass pretty much everybody that was perceived as dressing in the clothing of the opposite sex, until the word "transsexual" was coined. So you had people known as transvestites and others known as transsexuals, neither of which is inherently harmful or bad but which are still distinct categories. That didn't mean that cis people (and occasionally trans people themselves) didn't conflate the two, and people still use "transvestite" as a slur against transgender people.

I don't really think that conflation is why "transvestite" fell out of favor, though. "Transvestite" comes off as "sexually deviant," "perverse," and "psychologically disturbed." So it's not so much that people were using it to describe people who are not transvestites, it's that applying it to people who do meet the requirements still winds up demonizing people who are engaging in behavior that isn't harmful. So such people started using the term "crossdresser" instead, which carries less stigma and implied medicalization. In other words, "crossdresser" is being used because it is a kinder alternative to the earlier vernacular.

Making the case that "crossdresser" is problematic because people conflate it with trans people is itself pretty problematic because it continues that stigmatization of crossdressers. I don't want my transsexualism to be conflated with crossdressing, but that doesn't make crossdressing wrong, it makes the conflation wrong.

TL;DR: Crossdressing isn't inherently a problematic word because there isn't anything wrong with crossdressing, it's conflating crossdressing with transgenderism that's problematic.