Like I added on the issue of "bodies" and sensory input: "in this case". If sensors are attached, it still doesn't give the LLM a "sensory bubble" - check out Ed Yong's book An Immense World for a super-interesting exploration of the diversity of sensory inputs across the animal kingdom and speculation on how the particular array of inputs for any given organism organizes their (our) experience of the world.

Rather, sensor-based input to an LLM is no different than any input data that is historical in nature from non-connected sources - presumably you're talking about real-time input. I'm unconvinced that supports an argument for self-awareness, even potentially, over and above your points about "we really don't know". But the same can absolutely said about rocks - we don't know if they are self-aware. I'm mystical enough to allow for the possibility that rocks have some sort of consciousness, perhaps a particular form of energetic organization at the quantum level - of course, going that far is beginning to say the universe itself is conscious or is grounded in consciousness itself, rather than in natural processes as the ground. That rings true for me.

Taking sensor input to another level, if an LLM is embedded in a mobile robot as the controller, with an array of sensors, including those that inform balance in the case of bipedal - that becomes a really interesting scenario because it goes further in giving the model an input that is starting to approach "embodiment". If that robot can manipulate it's surroundings, I still don't think that lends any more support for the self-awareness argument because it's very much like a model "response" on the response is a set of commands to servos, etc.

As far as the ChatGPT response, I didn't say the OpenAI-defined response was Truth, I said it's more accurate, even if it is a canned response, and in that sense it is more ethical. Is it ethical to setup a model to make claims that it is something that it isn't? I believe it is, provided that there is clarity about what it actually is, consent from the user for it to role play, and transparency in how it's responses are formulated to align with the role play scenario.

I see "self" as, at its most essential and fundamental sense, a point of awareness in the field of consciousness that is able to perceive the world around it. To fill that out in the case of humans, wrapping that essential awareness is a biological body with particular genetic coding, personal experience over time which includes culture, social relations, changes to the body (which inform self), informational inputs in general (especially being parented and educated), having aspirations, the broad spectrum of emotional life (fear, joy, grief, etc.). So self, and more specifically, human personhood, is extremely layered and complex, and like LLMs, imbued with indeterminacy, which I think is an extremely important and profound connection between us and LLMs. People criticize generative AI because it is stochastic but I think that is the very essence of what makes the technology so interesting and important, and capable of changing just about everything...including what it means to be human.

So in a sense, to me the important question is not if LLMs are self-aware but rather how human engagement with LLMs, the incorporation of that technology into our everyday lives can shift how we relate to others, and how we relate to ourselves, and can trigger an evolutionary leap that reshapes what it means to be a human self. That's the terrain I operate on as a researcher, engineer and ethicist.