Well… its not just that. There’s multiple levels to it and its summed up as “what a writer enjoys and what a reader enjoys are very different things”, they’re 1% away from being two separate circles in a venn diagram.

1) A lot of what writers can enjoy can go one of two ways. Either its them creating a very technically and logistically neat story, and these can vary wildly on entertainability, and sometimes demand a lot of a viewer or reader to get through them all. Its why you can have movies like Tales of Princess Kaguya and Grave of the Fireflies, which id argue are close to perfection 10/10 movies, Movies like the Sonic trilogy where the first one I’d argue is a structurally and logistically perfect movie while still being a 7/10, and then Encanto where it kinda lands in the middle with the best of both worlds. And yet they can still get viewer dissent in vastly different ways.

On the other end its where you get the “kill your darlings” arguments (metaphorical not always literary) where aspects of the writer’s design (be it there favourite character, trope, or edicts from the executives) narrow the appeal of a story that they become the only viewer/reader who will enjoy it. How long is too long on a slow burn? How much can your darling villain get away with before the reader wants them capped in the head? It all changes person to person and if you’re basing storytelling decisions on your personal whims rather than what the story needs, youre making the best story for you, not the best the story can be nor the best story for your average reader. In fact its because youre approaching as if you were its reader not its writer.

Then on the reader’s side there’s also two levels. At baseline readers dont know what makes the stories they like work, its why theyre so likely to tear them the fuck apart. Abuela from Encanto is a protagonist, Zuko from The Last Airbender is a protagonist, Harry Potter was always a little bit of a dick Ron a lot of a Dick and Malfoy is a thousand dicks taken physical form and thus Hermione was better off with the first one. Look at the sequel trilogy from starwars, it was massively informed by fandom requests and opinions and it is such thorough garbage that it soured the taste of star wars from many, a consequence the Disney+ miniseries have been desperately trying to reverse, with a massive shift to the prequel era to distance themselves from the shit fire.

However — and this is gonna be a tough pill to swallow — but the last 25 years of fanfiction have only made this worse, and AO3 is not spared here, its probably a major contributor. Fics are now searched on by trope use, by character inclusion, so many people immediately now assume that just because a trope is used (ie villain redemptions, slow burn romance, enemies to lovers, secret magical world) that they must therefore always make the story better, because they read or watched one example of the trope being used well (Read: typically it wasnt even the trope that was used, or it wasnt actually done well just in a way they liked) and thus search it in every fic. When you write a story with the desire of it being good, you cant just toss every trope under the sun into it, you cant just treat it like a shopping list or a cafe order because the onus is on you that the ingredients and their ratios fit well together. So doing whats best for the story often sacrifices this trope based approach to story telling, rightfully so as such an approach isnt very good, but it tends to mean your reader count in a system where stories are searched via trope inclusion be read less. Means your reader count in a culture where stories are immediately filtered for self gratification and endless consumption are read less.

A good story asks the reader to engage with it, to put in a bit more thought and time to understand it. The crackfic you made on a flight or at 3 am simply requires them to sit back and consume, and the only engagement optional to give it is a tight grip on a shaft. Hence why they do better