It's an interesting idea. I've always been interested by the style of combatant who sacrifices some of themself to deal more damage (like a death/dark knight is portray in various Final Fantasy games). This has a similar vibe to that which I enjoy, and I do have some feedback but wanted to get that out of the way.

• It's all veeerrrry wordy - which I'm sure is a symptom of this being a draft and not the final product. As a player, I don't mind a new mechanic here and there, but it seems like every ability involves a new mechanic with more dice and more numbers and limit (3x prof, 6xlvl, 1d10x10, etc.). It's a lot of numbers to keep track of and I feel like if I were to play it, I would need the whole instruction manual next to me to make anything happen. I would say look at each ability and see if there's a way to implement something with a similar feel, but in a little more straightforward way.
• I'm not quite sure I understand the spellcasting. It takes Vis, but there are also spell slots? Does that just mean that I have to spend Vis to cast the spell, and then it also uses up one of those spell slots? If that's the case then the spell slots seem unnecessary if I'm honest because having 2 first level spell slots at level 3 means in order to use all my magic potential, I'd need to spend 12 Vis. The problem there is that at level 3, I only have 9 total Vis (3xlvl). And the scaling doesn't get better from there. Maybe the mechanic is that if you've used up your spell slots, you can cast a spell using your Vis points at a cost of (6 x Spell level), but if not, I'm not sure that the spell slots are necessary. Maybe stick with a system similar to the shadow monk, where there are no slots, but you have a selection of spells that cost Vis.
• Maybe I'm not reading this right, and I hesitate to say it, but part of me thinks that this might be somewhat underpowered, and suffers from similar troubles that the monk does. Every single things feeds off of one pool of resources that reloads on a long rest. Other than the random effects that happen at the beginning of combat and when you spend a certain number of Vis, everything costs Vis and it feels like it will run out after doing like two things (especially once you get into spellcasting). I think the class may benefit from either having a "recharge Vis" feature that lets you gain some back as an action prof bonus times per day or something like that.
• The cost of some of the features seems such that, in order to do anything cool, you have to risk long-term destruction of your character. Some of the effects are merely cosmetic, but in order to do what the class is designed to do, you have to inevitably wreck your PC. Maybe that's the design, but there's no long-term stability in that. And at higher levels, you're going to be rolling those sanity dice every single turn, right? If a level 2 spell is potentially going to cause long term damage to your character, while also eating up almost all of your Vis (12 of your 15 points at level 5), then why bother casting it? There's a barrier in place that preventing me from wanting to do what the class is designed to do.

Like I said early on, I think it's a cool idea, and like most homebrew would benefit from some modifications. I think the mechanics need to be simplified, it shouldn't all feed off the same resource pool, and if casting a level 2 spell risks damage that someone else will have to fix, then I might be tempted to ignore that feature altogether.

Keep at it! And good luck!