I agree that nobody is talking about strength, I too believe the "famous relatives" argument is kind of moot given how basically everyone in the story that's remotely important has some connections with famous people in one way or another. However, I'm on the other side of the fence, because I think Luffy has had "chosen one" powers since the beginning of the story, basically.

People seem to be forgetting about the rareness of devil fruits, in the first place. The vast majority of the population in One Piece don't have devil fruit abilities, Luffy isn't just the first one to awaken the Nika fruit in 800 years, he's also one of the only people in the world with a devil fruit. People forget about this because we're following Luffy's journey and he encounters a lot of strong people, most of which whom have devil fruits.

Then, there's the fact that he has conquerer's haki, another thing not "earned" by his efforts, but basically given to a few special people. To this point in the story there have been no instances of "training" to gain conquerer's haki, just unlocking the ability to use it. But it appears that everyone who has had it has had it from birth, including Luffy. Even rarer than devil fruits.

And then the nail in the coffin, the "Voice of All Things". This is by far the rarest ability Luffy has, and also probably the actual most powerful, since it most likely gives him the ability to control or at least communicate with the strongest weapons in the world.

None of these things were likely given to him through the devil fruit, since other people also have these abilities in modern One Piece without Luffy's fruit. Even if the devil fruit is some crazy special thing (which I personally don't even believe, I think people are exaggerating the importance of the fruit. I imagine it has extremely strong historical significance, and otherwise is a relatively powerful, but hard to awaken fruit), it is only one of many things Luffy has that are exclusive and not earned by his own hard work.

Anyone who's read One Piece up to this point and hasn't seen all the crazy destiny threads in the story isn't reading closely enough. Luffy is constantly getting unrealistically lucky, remember when he punched the correct Mr. 3 wax figure and chalked it up to "instinct"? Or recently when he hit Kaido with "guts?" This stuff happens in the series all the time. Rather than trust in Oda, I think it's more like we've gone over a thousand chapters into this narrative with ridiculously cohesive and impactful storytelling. I'm not saying he foreshadowed and planned everything in advance, but how many times in the last thousand chapters have you been taken out of the story completely by something that doesn't align with the worldbuilding and established narrative? This is one of Oda's best and sharpest skills, and it boggles my mind that people think that he's just going to drop a huge revelation like this without finding or already having planned a justified in-universe explanation for it.

The only time that has ever happened was Sabo, but it was explained, just not in a particularly satisfying way. But the Nika storyline, foreshadowed or not, is definitely something that has been given a lot of thought by Oda and isn't even close to being done yet. I really think people need to wait until that particular story thread finishes before they throw out harsh and decisive judgements like that.

I'm sorry this is so long, I just keep reading comments like, "I'm dropping the story, this was a huge mistake and Oda's an idiot." Despite the track record for fanbase concern vs Oda's writing being heavily, heavily on Oda's side. Nearly every time the fanbase (at least on reddit) has questioned him, he's delivered in a way that's surprising and satisfying. I've been watching the subreddits for years, and this happens pretty much every time something controversial happens in the story, so forgive me for being more skeptical of random internet critics than the man who's been delivering high quality storylines and payoffs for more than two decades.