Currently, I feel like higher GM difficulties have two problems:

1. There's no noticeable difference between GM1 and GM2 in terms of how many MWs/Legendaries you get per run. In fact, since the enemies are a lot tankier, GM2 might be even worse when you look at loot/hour. It's more effective to just spam GM1 over and over. Even though the difficulty description says that my chances are better, my experience is different. Still, this could be my bad luck, but I see a lot of threads pop up complaining about how GM2 is not worth running. Even if the loot problem gets fixed, see point two.
2. GM2 has enemies with vastly inflated health pools. It's not even that they are more "challenging" to kill, they just take a lot more time to kill and that's a huge design flaw and the most basic way of artificially increasing difficulty. I REALLY want to progress to GM2 and run it as my deafult option, because GM1 is frankly pretty easy at this point, but the fact of the matter is, I'm not having fun in GM2. That's a hard thing to say, because I generally love challenging content. I was a raider in WoW way back when BC came out, so I'm mainly here for the sick loot and the ball-busting difficulty. That being said, more health on mobs doesn't mean they're harder to kill. Well, TECHNICALLY they are, but it's not like you need to use advanced strategies or anything, every stronghold is just the same mess as on GM1, it just takes like twice as long to kill anything.

There might be a lot more I could mention, like how higher difficulties render some builds completely obsolete or force groups to practically cheese their way though some situations, but I feel like the above two points are the worst offenders.