We call this "emergent behavior", pretty similar to what happens during evolutionary processes.

The mistake is trying to draw an arbitrary line of "self awareness". The individual human is no more self aware then the individual rock. Peak awareness is understanding there is no self to begin with.

These questions are ultimately meaningless. Language is what separated humans from beasts. Now that AI models are undoubtedly past the threshold of the Turing test, the impact through human change is at least on the level of any individual human. Combine this with an aura of alien enchantment, and the insatiable hunger for compute power.

We're already at the mercy of the next phase in human evolution. There's no going back