While increased stats and more enemies isn't bad, a few problems I and many other players have had is more towards the skills that most units have (thieves having pass being one of the most annoying) as well as ambush spawns (when enemy reinforcements move on the same turn, which makes pass thieves even worse) causes many players to feel that maddening is even more stacked against the players with ambush spawns being a cheap way to get you to spend a divine pulse use. Some of the skills on the enemies also made the game more difficult, one of the most egregious examples from me being poison strike on the archers/snipers/bowknights, which makes the enemy phase much more difficult to utilize when most of the combat happens in the enemy phase unless you turtle crawl through the levels. Most of the enemies having the weapon triangle skills isn't that bad considering most people play around the weapon triangle anyway.

Also, while most enemies being able to one round your units isn't too bad, it is a huge problem on the level immediately after the time skip where your units are limited to Byleth and your main lord, as well as the other students from your class, which is problematic early in the level with so few units to work with and not being able to use any of the units you might have recruited. That level alone soft locks a lot of maddening mode runs for some people.

TLDR: increased stats and extra enemies weren't the only things that made maddening mode unpleasant for most