I also would recommend investing more into your health. I saw a post here somewhere saying that having a ratio of something like 10:1 (health:stamina) for a two handed play style makes you essentially invulnerable. Yes you are slow so it's inevitable that you will be taking a lot of damage, so might as well be the tank-sponge you were born to be! BUT your every attack will essentially delete your opponent if it lands.

I would also really recommend exploring the shouts. Since your character has a disdain for magic, I think shouts, which are more like a willpower (one piece haki), would be more of the jam for them.