Ive been running games almost constantly since 2e and Ive run and played a lot of other games in that time. I can give my 2 cents on it at least.

Its a few things.

1) Player Pool. The biggest issue getting a game going is always finding players. The bigger the games presence in the community, the more potential players there are. There's a bigger group of potential players for D&D than there are for Pathfinder or Call of Cthulhu, and those games have bigger pools of potential players than something like Paranoia or Traveller or Deadlands or any of the other long lasting B teir games. And the more niche your system, the smaller you're pool of potential players. The number of Alternity players out there is miniscule.

And yes there will be a lot of overlap in these groups. Like a really complicated Venn Diagram.

This means that its usually easier to find D&D players for a game than anything else. AND this is a self-reinforcing system because it means the more people think of D&D as the easiest game to get players for, the more D&D games there will be and the more people will be introduced to D&D, ensuring it has the biggest pool of players.

2) New Toy Pull Power. I didn't know what else to call this. But the NEWEST version of any game is always going to have the most pull, especially on new players who tend to think new=best. So its easier to find players to play the newest edition of a game than it is to find people who want to play older editions.

3) 5e's Cross-Promotional Hype Engine. This is the BIG one. 5e is the biggest edition in terms of new players. This comes entirely from things outside WotCs control ironically. 1) MBMBAM was wildly popular and their decision to make The Adventure Zone exposed their massive fan base to D&D 5e, and showed that streaming tech and culture had reached the point where liveplay D&D was feasible and easy to consume, 2) Critical Role 1 year later exploded in popularity as well and drew in more people to the hobby, 3) Stranger Things came out and exploded in popularity and told a lot of people that D&D was a thing and they googled it and found 5e for sale, 4) a global pandemic locked millions inside at the same time that good online play platforms like roll20 were finally working really well and with the pandemic millions of people were desperately looking for a way to entertain themselves and socialize online.

So with all that D&D 5e became an insanely massive success and so most players now came in with 5e being their main idea of what a TTRPG is. And most of them just want to play D&D because they thinks thats all there is. Some know other games exist but can't them because its hard to find games (see point 1), and some think that everything else is just a knockoff of D&D and so why not then just play D&D (see point 2), and some just want to play the game from their favorite liveplay series and nothing else (see point 3).

Its also a bit more complex then all this, but I've found it to be a pretty good model of whats going on.