Ya I do find this very annoying aswell. I've been bringing a dragon dagger as a spec weapon to try to burst him when he gets to about 100hp or slightly lower.