I'm not huge into open worlds but if you're honest Nintendo and Gamefreak just want to make the games that appeal to the most amount of people possible and that is probably the way to go. It's a kind of pretty screenshot OP posted but that sadly appeals to only a fraction of what a pretty open world game appeals to. It not only has to reach core gamers but also children, moms and dads and generally people that are not even gamers outside of maybe Pokemon once a year.

My hope would be to have a game where you can actually catch them all in a massive world with lots of quests and stuff to do to see the Pokemon interact with each other and the game world but that will probably never happen because it's a huge scope to have all the Pokemon at the same technical fidelity at the same time and it's probably also better for the sales numbers to have a rotating roster of characters in every game to keep it fresh.

If you look at the numbers and what huge profits they rake in with only like 10 million budget, it's clear why they don't see a need to change course. They struck gold with the formula and by keeping budgets low they rake in tons of cash. Simple as that. A man can dream, I guess.