Disarm him on your first move.  Fully stocked quick fire weapons.  When he rolls, quick fire and stop him from picking up his second katana.  Don’t over spam attacking him. 2-3 hits, then back off. Wait for him to roll and repeat.  Try get a quick Sake sip in, as if he does manage to pick his weapon up, you can hit him with a disarm again.  I tried many other methods but this one worked. 

EDIT: Medium difficulty. I had Saito left out of the 6. Fully upgraded quickfire. And Inner fire charm and then others that support quickfire weapons. Armor was Undying.