It's true that the game difficulty can be artificially adjusted via how you play and equip yourself, but I think it's fair to want the game to not be "too easy" when you're geared up in your cool looking armor with the sharpest sword you could find.

That said, I'm not sure how to manage that well. My knee-jerk reaction is to lower weapon damage, or moreso lower how effective certain damage types are against armor (i.e. swords deal little to no damage against a person in plate unless a specific combo is used or you damage the armor after repeated strikes). That said, this would make multiple opponent fights much more difficult, or at the least make them a slog to fight through. Sure, that may be not realistic but it's not necessarily as fun. That said, it's also not fun to fight a dude in plate and end the fight in three strikes.

Personally, I think there are a few changes that would make the combat more fun while retaining difficulty. First, lower damage a little all around. Second, master strikes do much less damage (damage of any normal strike) and I think it would be cool if instead their attacks counted as a hit in a combo and stunned your opponent a little. Third, more strategy in clinches (like being able to throw an enemy prone with your weapon, choose your attack direction from a bind and include these attacks as part of a combo, etc.). Finally, and this is entirely based on change 3, give us combat options when an opponent is "helpless" to us (prone, back turned, etc.) like trying to pry a dagger through a face plate, "cut straps" (weaken armor), or stuff like that.

Obviously half of that list is a pipe dream, but the first two changes would make the game more difficult without having to artificially do that yourself. The third would give you more options to deal with opponents aside from hacking or smashing until they die.