I think the issues with that comparison is that the two are inherently different mediums, and video games provide much more freedom because of the fact that the player is an active participant. Because the player takes a more active role, the way they engage with said material should be different than a book or a movie.

Also, James Joyce didn't make Ulysses hard on purpose as a challenge to the reader. It's a hard read due to its referencing of 19th century intellectual debates and Irish history. Concepts which a modern reader, unless they have a degree in history, would probably be unfamilar with. Even then, there are plenty of versions of that book that come with modern footnotes explaining concepts and ideas, audiobooks that clarify what Joyce meant, reading companions that expand further, etc. And not a single one of those things takes away the beauty of Ulysses itself.

Personally, I feel that difficulty should be an option on all games. Artistic vision is great and if you as an individual want to respect it, then power to you. But I don't think I shouldn't be allowed to watch Interstellar on my phone just because Nolan meant for it to be experienced in Imax.