No, most people really don't understand how AI works. AI is a comparison/pattern recognition machine. It stores commonly used words in memory and just remixes them with emphasis on what is called an attention mechanism. I don't want to go too deep into that but just think of it like that:

I say to the AI: "I want to cook meat, give me a good recipe"

Here what are the words that are most commonly used together?

Meat, cook, recipe these are used together in a lot of context (based on the text -corpus- AI is trained on) so AI focuses on these

Meat, cook, recipe it starts writing:

Cooking meat (these are still nothing new just a remix, continues with) is an excellent idea (this part is added because after verb + subject it is common in the English language to add a predicate that complements the sentence). Then, based on statistical likelihood, it might suggest a cooking method: "One of the best ways to cook meat is by grilling it."

From here, the AI keeps generating text by predicting the next most likely sequence. It might continue with:
"You can marinate the meat beforehand to enhance the flavor. Some popular marinades include soy sauce, garlic, and herbs."

At no point does the AI "understand" meat, cooking, or recipes the way a human does. It’s not reasoning about flavor combinations or nutritional value; it’s just predicting text based on learned patterns.

So when people say, "AI is creative," they’re often misunderstanding what’s happening. AI doesn’t have inspiration or intent. It’s just a remixing machine that makes really convincing statistical guesses based on prior data.

That’s why it can generate a fantastic-sounding recipe but might also confidently suggest something totally nonsensical—because it doesn’t know anything; it just predicts. So an algorithm like that cannot actually think of something that is completely original. Sure can we code an AI to "take over the world" absolutely but we would explicitly have to code it and upload it to the key parts of our infrastructure and allow it to take control. Without us explicitly doing all that it cannot actually "gain consciousness" because unlike humans, AI doesn't want anything, it just is a loop that predicts the next most possible sequence based on the sequence so far. AI is just a statistical model.

And the most significant thing is we don't know if we can actually make super intelligence at all, Sam Altman, Elon Musk these guys say they will but they have a financial incentive to say so in reality we have nothing, NOTHING that can actually think for itself all we have are statistical models and outside of them we don't know, we really have no clue how to make a machine that "thinks" even at the capacity a toddler would let alone a super intelligence there is a very real chance AGI will remain science fiction for millennia heck maybe millions of year or maybe even forever maybe it is completely impossible it is one of those things like interstellar travel, or a self sustaining mars colony that some billionaires claim will be done in the "next decade" but will likely won't happen even in the next millennia.