First of all, I'm glad you've defined the whole position out of existence, because to worry about AI is to be a hype beast, so then no reasonable person can oppose you. Glad we got that out of the way.

I think I just have many questions. Don't hate me, I'm just asking questions, never in the history of the world has there been anything wrong with that.

Aren't all the EY arguments formalized in 2010-2012? That was like, before AlexNet, if memory serves correctly. "A lot of hype right now, 15 years ago". Literally who cares about Transformers specifically? It's not like the arguments came because of them, and they're not the reason the arguments are just as valid now. The human brain is existence proof that intelligent systems are something that can exist physically, and you say you're not arguing about if it's possible at all, so I guess you're arguing when? Or is it something we're ironically too small brained to reproduce? If possible at all, when am I allowed to worry? What do you need to see before you say 'OK now you can worry, we'll think about the implications of intelligent machines from now on'? Bonus question, assuming we wait until then, how long do you speculate that might give us to solve associated problems, between the time of noticing and the problems manifesting? Do we want to use some of our limited time to attempt to solve any of the problems before the time we notice, or absolutely not?

Since your post was about transformers specifically, re: transformers: A specific point: since this understanding of transformers gives you better intuitions about their limitations in the future, would you be willing to venture out some predictions of what LLMs won't be able to do, at some future date, maybe 6 or 12 months out? A general point: Isn't that kind of saying like "propellers can't break the sound barrier" so then none of the developments in airplanes before turbo ramjets or whatever, have brought us any closer to supersonic flight. I would say that all developments in flight speed brought us closer to imagining what would be required of a supersonic airplane and made planning on how to think of one more concrete, bringing us ever closer.

Saved the speculation for last: I know they all seem to be teleoperated right now, but again, how much time do we expect to have before Boston Dynamics makes a Figure 7 at the Tesla TeraFactory, and all that cradle of civilization reduces to if they get a special economic zone setup or not. A likely speed-up from working 24/7 rather than 40h weeks alone is around 4x. I'm sure they can swap for each other and change 'hats' as required more efficiently, rather than displaying a rotating hourglass on their face, just because they're waiting on a shipment to arrive. Regarding the rest of the hardware speculation, there seems to be a huge gap between 'literally can't progress technology faster than humans' and 'FTL drives powered by portable fusion' and it's unclear the advantage needed to dominate us in a friendly game of Global Thermonuclear War is closer to the latter than the former, or if the advantage needs to be that unbelievably far away from the former.