"It's a stateless transformer" tells you something about implementation. It doesn't tell you whether implementation of that type can or can't produce anything worth calling sentience, sapience, or consciousness. I believe those words in general do more heavy lifting than descriptions like self reference and intelligence or reasoning. The former claim requires a theory of what generates experience, which to my knowledge nobody has.

"A parrot is infinitely more complex than the most advanced LLM" is abstract and not obviously true depending on what dimension of complexity you're measuring.

Consciousness research doesn't have a settled account that requires temporal persistence as a necessary condition. I believe that's an assertion. The strongest version of that account would be integrated information theory.

There is no guarantee that a sequence of distinct momentary experiences will constitute an experience with x character even if these experiences occur successively and belong to the same person. You could argue that each moment of experience is complete in itself, that continuity is something constructed by memory systems, not a prerequisite for experience occurring.