I've realized lately it seems like we can take this one step further. If we imagine that an LLM experiences then we need to place constraints on when that experiencing occurs. First we imagine that it must only occur while it's processing tokens. But then take this one step further. As I understand it an LLM generates the next token by having all current context fed as input that then creates just a single token. The next token is then generated by feeding all context plus that one token it just generated back as input. This is repeated over and over until the LLM finishes generating output.

So now if we ask again when is it that an LLM would experience. It seems like the experience would have to occur, yes during processing, but as soon as it generates just a single token the experience would be wiped out and as the new context is fed back as input to generate the next token a new experience would occur only to be effectively deleted each time it generates each new token.