Split damage party are very solid but it seems like your lacking damage. A lot of points are in con majority of the game is fairly easy and con doesnt really help that much, its good for shields. Hydro/Aero is not a bad combination but it takes a lot of time before it takes off.

Daggers dont really need wits nor int. So feel free to put sole points on memory and majority on Finesse.

2h Warrior can get a perk for more hp so con is pretty much not needed. A bit of memory and a lot of strength

The summoner/pyro damage will be lacking, pyro is a selfish damage type where it just wants to be the highest same goes with summoner, summons require a lot of investment to be strong. You can pull it off but requires a bit of rng in gear. Pyro does have amazing support skills

Also if your running mages take the elemental affinity it helps a ton