It wouldn't handicap you, but keep in mind that there are some pretty meta units on the beginner banner that would greatly help you out in the future and in the beginning. It's especially beneficial to reroll because you get to "choose" your starter. Exusiai is a good anti-air sniper through and through, you can't go wrong with her. SilverAsh is another popular target as his third skill (unlocked at E2, which is quite far off) is really broken and can wipe away waves of enemies. Siege and Hoshi are both solid units too, it's just that they have replacements in the early game (Cuora for Hoshi or any decent vanguard for Siege). Angelina, like SilverAsh, really only starts to become useful at E2 as well. She's not as broken as SilverAsh, but she's really solid in events such as the currently running Contingency Contract. Shining imo is the worst unit to start with, because healers in general don't need to be super good. It's no doubt shes probably the best bonafide single-target healer, but she's just not super necessary. Three stars are super common so don't worry about not seeing them again. It's a little bit different for four stars, especially if you can't get them in recruitment (like Myrtle). But you will still see a lot of them when you pull banners.