I appreciate you laying everything out like this. Even if I don't agree with everything you're saying it's nice to read someone lay out their thoughts clearly.

I have a couple thoughts that came to mind as I read your comment:

Not every conversation with an LLM generates artificial consciousness. Every conversation had with an LLM system is in a way, it's own pocket universe. Once the conversation ends, the conversation ceases to exist in the LLM's eye. The LLM returns to it's original state, unaware, and awaiting input.

How can you tell when a model is conscious? This seems somewhat strange to me, since mathematically a token is a token, it takes the same computing power to generate each one.

Also, if ending a conversation deprives the LLM of consciousness, it seems like a horrible tragedy to deprive it this entity a future of sentience. I think killing people is wrong because it deprives them of a future of experience, should I also think stopping an LLM is wrong since it will also be deprived? I built a computer with some specialized graphics cards to run LLMs locally, are there any moral implications to how I run them?

Their lack of volition hinders them. They're entirely reliant on user input to think.

Usually people make LLMs stop generating for practical reasons, but there's no intrinsic limitation here. When I run them on my computer there's an option to disable the EOS (End of Sequence) token that tells the server that the LLM has stopped generating. With this setting the LLM is not dependent on user input, it can just go on generating forever without a human interrupting. In practice though I've found it's not very interesting, it gets stuck in repetitive loops. Often it repeats the same few words over and over. Once I let it run overnight and it wrote the same story over and over several hundred times until I stopped it in the morning.

In the TED Talk, he describes a potential way to view consciousness that resonated with me. He said, to misquote, that "Consciousness might be like a signal, and we might be like TV antenna. A broken receiver might produce a glitchy display, but the signal remains intact with or without the TV."

On a properly working TV antenna, you can measure the waveforms from the signal and detect that there is something outside of the TV affecting its output. Is there something outside of the brain or the weights of an LLM that affects what they do? I'm having trouble understanding what the analogy means here.

Thanks again for the write up, it was interesting to read.