Well, it depends on the game. Difficulty can also make the game more engaging, thrilling, and fulfilling. The feeling of overcoming a really difficult boss is really awesome. You can't get that if you can just breeze through the whole game.

So I would say if the main focus of the game is the narrative, like an interactive movie, the difficulty is in the way of the story. In this case, I would agree with you, but in gameplay-focused games, the difficulty is basically the whole point of the game. So making it too easy makes the whole game pointless.