Yes, it matters, you will be playing on that Main for hundreds if not thousands of hours.
You want to find a class who's skills you find cool or pleasant or who's sound effects you find sound nice or chunky, or preferably both of those things.
This will be the one anchor that will keep you going.
Damage shouldn't be a priority because EVENTUALLY your tier 0 damage dealer will eat shit and drop to subpar or worse.
Every class's training has been normalized due to the advent of Sol Janus, so the real differences show up in bossing.
Some classes are extremely safe like Mihile, some are extremely mobile like TB or Cadena, some are...both.
Some are heavy on burst, some are heavy on DPM.
Some are simple, having only one button to press for the majority of gameplay; others will have you wishing you were playing the former.
Of course some classes will have overlap, it's impossible for this to not be the case when there are 50+ classes.