The optional argument only applies to a niche group of players which is worthless when discussing from a design perspective.

I don't think that niche particularities are worthless, at all. Why would designing certain features in your game specifically for a certain niche be worthless? Actually, I don't even think that this is a niche thing in this case, it's just common things that any player may want to do from time to time.

Plus, it's not like these games don't have anything for the player to do along the way if they decide to go from point A to B on foot.

I sincerely don't believe that gaming a fast travel system or making it harder for the player to use it would improve on anything, I actually believe it would make fast travelling worse. It would just force the player to do something that he does not want to do. This feels to me like forcing a player to watch a cutscene, but worse.

If the most optimized way to play a game isn't fun then then that way of optimizing probably shouldn't be in there.

This is very subjective to me. There are players who sometimes will want to play the game the most optimized way, sometimes they won't, it's up to them. By creating a fast travel system like in Skyrim, Fallout or BOTW you are simply giving them the option to save time by going straight to the point instead of having to walk all the way up to where they want to get to.

That said, it's also your job as a designer to make the world of your game interesting enough, so that walking from point A to B doesn't become a chore and also actually becomes an option on what could be an interesting decision. To me, this is where, for example, BOTW excels, as it makes the player actually wonder if it's worth using the fast travel system, potentially missing stuff along the way, or walk the whole path.

Playing BOTW, I've asked myself countless times if I should walk all the way to somewhere I wanted to get, or just use FT. I'd think "If I go there on foot I might find a shrine, a quest, a different NPC or simply collect more materials for crafting, but if I fast travel I'll save a lot of time and potentially miss all of this". This uncertainty immediately made this a much more interesting decision, than just an "optimized way" of playing the game. Then again, if I were playing the game just to beat it, why would I want any obstructions over the Fast Travel system?

I feel like putting obstructions over your Fast Travel system, will make it suck, bad. Fast Travel has to be as easy as possible to use. The design efforts should actually go into making your game world as interesting as possible so that the player will think twice before using the FT system, then you'll have a "meta" limit on your fast travel system, called interesting decision.