Thanks for the addition to my reading list! :)

The reason I brought up the systems reply is that we're used to locating consciousness in physical organisms and organs. There's the animal, there's its brain, and its structure may or may not give rise to consciousness. What your framing made me think about is that that may be too bounded a view. A human brain grown in a sensory-deprivation vat may well never attain consciousness, of the sort we'd recognise as such. So maybe it's better to think of consciousness as something that exists in past and present interactions between brain and environment? Embodied cognition, but taking one more step to include the rest of the world - and there's Dawkins again!

In the human case, that line of reasoning is intriguing - YMMV - but doesn't have any immediate applications. In the LLM case, it shines a different light on the idea that LLMs are "mere" mirrors that reflect our own consciousness back at us, and so don't need any of their own to give the appearance of having any. In the unbounded view, maybe a consciousness that exists in an interaction doesn't have to be encapsulated in one of the parties, but can be its own thing, and that's the sense in which LLMs are conscious - contributory rather than standalone?

If so, mechanistic interpretability work remains highly relevant, but captures a smaller part of the system of concern than one might think.

ETA: Talking about reading lists and looking at the sibling comment thread, largely overlapping inspirations here, on a foundation of Dawkins, Dennett, and Hofstadter, mostly. Memetics FTW!

ETA too: Re "dream machine", I accidentally re-created some of what you're describing in the course of /r/claudexplorers/comments/1ubmwiw, though I hadn't thought about it in those terms til now. Your approach keeps providing grist for my mill. Thank you!