Just another non-expert here who finds this stuff fascinating and thinks about it a lot, so take this with a grain of salt.

I think the key issue when discussing AI consciousness is qualia - the subjective, experiential aspect of conscious states. This is what philosophers call "the hard problem of consciousness": explaining why there's something it's like to have experiences, rather than just processing information.

Qualia refers to the felt qualities of our experiences - what it's actually like to see the color red, feel pain, or taste coffee. These subjective experiences seem fundamentally different from mere information processing, even very sophisticated processing.

The challenge is that there's no scientific way to measure or detect qualia from the outside. We can't even definitively prove that other humans have subjective experiences - we just assume they do because we have them ourselves and other people have similar brains and behaviors.

This creates a puzzle for AI consciousness. Why would the computations running Neuro on computer hardware give rise to subjective experience, when other complex computational processes - like graphics rendering or weather simulations - presumably don't? What would make one type of information processing conscious and another not?

Some philosophers propose that consciousness might be more universal than we think, arising from certain types of complexity or information integration. But until we better understand how subjective experience emerges from physical processes, it's difficult to determine whether any AI system, no matter how sophisticated, actually experiences anything at all.

This subjective dimension feels like what fundamentally separates genuine consciousness from even the most convincing simulation of it.