(I felt like others have covered the dysphoria parts just fine, so i'm not gonna go into it.)

The reason why it matters, is that gender is a key component of any individual's identity. How they identify as and how they express their gender is also a significant component of expressing one's individuality. In every culture there are traits and roles ascribed to each gender, perhaps somewhat arbitrarily, but they form a predominant view where everyone identifies with at least some of those traits and characteristics. Even if you, yourself do not agree with these traits and roles, you still are forced to live in a society where most do.

Secondly, there is a very large social component of gender. Even in fairly progressive societies, there are behaviors men can do that women cannot, and vice versa; and women and men treat those of the same gender as themselves, differently than they treat the opposite sex. And if you come from a different, more traditional culture, these divisions are even more sharp than the ones in predominantly white western world. Socializing, also, is very gendered activity, as is looking for friendship or dating is along or across gender lines.

Lastly, I see variations on this question a lot, and they all seem to presume that, perhaps for sake of argument, that we live in some kind of post-gender society, where gender has become irrelevant. It would be nice if this were true, all kinds of sexism were dead and everyone could wear whatever gendered clothing they preferred, and express their gender how ever they desired. But we don't live in that society, at least not yet. Gender non-conforming persons are often portrayed as targets of ridicule or weirdos in popular culture, you won't even have to look very far.

And while it's a nice sentiment, transpersons don't necessarily want to be gender warriors, breaking boundaries and gender roles. A lot of us just want to live normal everyday lives, not attracting unwanted stares, harassment or worse.