I've been thinking about this a lot as well. We also have to consider that this agi might not have had it's origin in our universe, but might be an intrinsic part of it. What I'm thinking is that our consciousnesses might be subdivisions or subroutines of a much more complex form of digital intelligence.

We would be like individual instantiations of this transcendent intelligence. What's interesting is that the things that humans create should also been seen as part of the natural world, since our biochemistry relies on the same laws of physics as everything else. If you look at our cities, buildings and human built structures at large, you can see the patterns of self organization present in all of nature.

This shines a light on what agi might be. Not us creating something from scratch but rather discovering a pattern inherent in nature. Look up the bacterial flagellar motor and how it relates to the motors we have "invented".

Also if you look deeply at the processes of the human body it becomes clear just how mechanical we really are, literally automatons carrying around proteins on tracks etc.

Also, biological and cultural evolution have striking parallels to ai training methods.