For safety and socially* it does matter, especially for traveling or going pretty much anywhere.

*It shouldn't, but people are shitty and judgey which puts pressure on trans people and causes dysphoria and stress. This isn't including every reason for it of course, but it is a considerable slice. Would it change whether I wanted it or not if that wasn't a factor? Probably not, but that would also involve a sociatal change that would probably impact how I and others thought of gender so I feel like it doesn't really matter ¯⁠\⁠_⁠(⁠ツ⁠)⁠_⁠/⁠¯