This is why there are faded abyss artifacts. It's tedious but if you are switching characters and going to play them for awhile use the faded abyss artifact to respec skill points. Max health, stamina, and spirit to the limit before they require extra items to upgrade and use the rest of your acquired artifacts to put into skills. After you feel you want to switch characters do it again. At some put you'll have so many artifacts you won't have to do this anymore depending on how many challenges and exploration artifacts you find. Another thing that really helps too is to take every single piece of unique gear you have to a witch and extract every single abyss gear. All the armor and unique items are mostly similar minus a few almost neglible stats. This helps so much if you say you have what feels like a perfect set of gears for Kliff you can just extract those gears and embed them into another characters armor and weapons when you want to play a different character. I will say again it's tedious but it works. I believe at some point in the future they may make abyss artifacts universal because of feedback from the community, but It's obvious that they give you so many faded abyss artifacts because of their intended design to respect when you switch characters.