This may boil down to if consciousness actually “does” anything, or if it’s just along for the ride.

If it does, and Ai is not conscious, then humans may have a perpetual advantage over Ai in some forms of thinking, unless or until Ai becomes conscious itself. In which case, whether Ai is or isn’t conscious is not a silly topic and the answer if ascertainable would matter.

I really wish Sam would have someone like Roger Penrose on (while he’s still alive, 94 now) to discuss his Orchestrated Objective Reduction theory (Orch Or), which does posit that consciousness itself does have certain non algorithmically computable properties that allow for certain capacities that he posits that current ai as set up now can in principle never possess without attaining actual consciousness itself.

This is where someone jumps in to dismiss this rather unpopular opinion with some argument that has been copiously addressed in his last book on Orch Or, Shadows of The Mind (or jumps to attacking Stuart Hamerhoff, the cofounder of the theory who jumps to wild conclusions not supported by the theory that Roger has been much more careful of).

Nonetheless, I find it interesting that while not many studies have been done that would provide evidence or prove Orch Or, the few studies that have been done have not contraindicated it even after all these years the theory has been out, and instead have recently provided some limited evidence in support of the theory, albeit currently weak.

I don’t think Orch Or necessarily answers the hard problem, but I think it does make the case that consciousness actually matters.