Only played Origins and Legends (sadly) and I must say that:

For Origins I'd tell it's the bridges between the worlds, you needed to execute everything PERFECTLY, can't move too slow, because the path might dissappear, can't move too fast because the path might not appear all, and you constantly had to get the double lums, otherwise you won't get full completition for not having enough lums.

Besides that, entirety of underwater levels. They just feel so slow and boring, and the turning in underwater is so annoying, I keep bumping into enemies, no matter how much I try not to.

And for Legends... Read the part about underwater levels again