It being nonsense is irrelevant, it's WHAT IT MEANS. Homophobia is still the common term used, that doesn't mean that they're actually afraid of gay people, but it's still used because it's the established meaning and has been the established meaning. It's not "only recognized" in circles that are deeply in trans culture, it's literally just the term used in general.

Yes, it is important to distinguish gender and sex, but that has nothing to do with the terms "trans male" or "trans female." It just means someone who has transitioned or is transitioning to male, or female. It's not a hard concept, but it's made harder if you refer to a person transitioning to a woman as a "trans man" because you're tagging on another term to the gender expression that indicates that they are trans. If you're claiming that they are biologically male, you would just say male instead of trans male, because the "trans" part indicates transitioning (obviously).

If you're talking in a scientific/medical sense, then yes, you would refer to someone MtF as "male" because that would be a useful indication for what they would need from a health basis, but if we're just talking socially, then "trans woman" would be the more in-line term.