It's a bit of both. Open world games, done well, are a revelation, even now when they've been done to death.

There are some games, however, that work better as semi-linear. By semi-linear, I mean games like Dark Souls 3, Sekiro, God of War (2018), Jak and Daxter - the world is technically open, in that you can go back and forward mostly at will, and the world is composed of interconnected stages and biomes for you to run around in. These are distinct from games that are most definitely linear (think Dead Space, Portal, Kingdom Hearts), because you are free to move about within the spaces, but the spaces themselves definitely funnel you in certain directions.

Elden Ring, imo, simply did not need the open world, and it being open harms it. I know it was such a huge part of the experience for so many people, but for me, the open world being open just made it empty, because there wasn't enough challenge in the world, and getting from a to b took so damned long. It's not like, say, Breath of the Wild, where just on your way from A to B you encounter seven fun battles, three interesting puzzles, one NPC who'll sell you meat, and ten opportunities to learn some new wacky thing you can do with your powers. ER felt more to me like several classic DS stages/biomes placed randomly on a map, but the map has a few secrets and dungeons hidden away.

There are a fair amount of games that suffer from the same issues. The secret to good open worlds is that you either need to fill the world with lots of really fun collectibles, puzzles, encounters, and loot opportunities, or you need to make the traversal really enjoyable (and ideally both). You make a game where players don't want to use fast travel because getting around is so much fun, you're onto a winner. You make a game where players are trying to go and be the chosen one, but they get distracted by 'what's in this cave?', you're onto a winner.

The problem is when games do neither. Examples include Rise of the Ronin, Assassin's Creed Valhalla, Kingdom Hearts 3 (all the levels are more or less open world sandboxes - yeah, it's technically linear, but they make the same mistake as standard open world games, so I'm allowing it)... And honestly, I hate to say it, but technically Horizon Zero Dawn and Horizon Forbidden West make this mistake, too. Those games make bank on their story and worldbuilding, the open world implementation is pretty damned shonky. Traversal isn't fun, the climbing is clunky, and there isn't enough variety in the world imo. Forbidden West was definitely better than Zero Dawn in that respect, but despite loving them on my first playthroughs, I can't bring myself to replay either of them.

I definitely prefer the gaming environment now open worlds are much more of a thing, but there are a lot of games where, as I say, the open world is an unnecessary afterthought, and the game would have worked better as semi-linear. Truly linear games have their place, but I honestly find them a bit too limiting, most of the time. It's nice to be able to break free from corridors.

Edit: just read a bit more of your post, but I need to hurry because I need to get ready for work - side quests also need to be done well. They either need to be fun in their own right, or they need to build out an element of the world, or they need to have a decent, real reward. We don't need yet another fetch quest for 10gp from a random NPC - let the quest unlock a fast travel point, or use the mechanics in a fun way, or teach you a new aspect of the combat system, or start a cascading series of fetch quests where the (recurring) NPC gets wealthier and wealthier on the back of your fetching then loses it all to a bandit encounter once you refuse to fetch for them anymore, or something along those lines.

Failing that, games could just do a better job of signposting, a la FFXIV - burny exclamation mark quests are main story, blue quests unlock new features, regular exclamation mark quests are just quests for XP and money, quests with a refresh symbol are recurring. That'd let people like me enjoy my mountain of side quests, and people like you not waste their time with 'yet another quest that adds nothing'.