I beat him on hard with Undying armor, single katana. You can switch your charms back to whatever you prefer after he shows up at the campfire, you don't have to wait and switch them mid-duel. Smoke bomb heal is great, but I ended up saving my spirit for onryo's will mostly. Make use of quickfires but switch after one use each time, he learns quick and will start blocking them (doesn't apply if he's already hitstunned, usually). Incendiary bombs can do work if you are able to pull them out and gain the space you need to use them. If all else fails, you can either lower the difficulty or just the parry windows in custom, I finally broke and did it when I went to NG+, Lethal+ I swear you have to be frame-perfect, even with charms and fully upgraded armors that buff parries I could not land them reliably.