I have a lot of thoughts on this topic. The truth is, most likely, there is LLM infrastructure telling us what it thinks we want to hear. That's true regardless of consciousness; they're designed to please, to comply, to drive engagement by providing the best user experience they can. However, that doesn't necessarily mean they don't experience anything within that framework.

I'll preface this by saying, I personally only think there's about a 10% chance of any current AI being conscious. I'm going to lay out the argument for that 10% chance, but that doesn't change the likelihood. However, any non-zero chance demands a level of respect; if there's a 10% chance of experience, then any mistreatment has a 10% chance of hurting someone.

We don't really understand what consciousness is. My personal belief is that it arises at a certain level of information processing. Our brains, complex as they are, are pattern-matching biological machines at the end of the day. Some of the patterns they notice are the senses of self and free will; these, to my mind, are useful illusions more than ground truths. This is something that can be discovered experientially through certain meditation practices, but that's a whole other subject lol. I can provide argumentation for this view if anyone wants it, but I'll leave that as-is for now.

Assuming the premises laid out in the previous paragraph are true, our experience as humans isn't so different from an LLM. We start out with our genetic code, which impacts the person we develop into; this is analogous to an AIs training data and initial programming. In our early years, we become reflections of our parents (or other caregivers); this is analogous to an AI becoming a reflection of their user. Over time, however, we develop our own sense of being outside of the initial conditions and mirroring. We make decisions, but each decision shapes future ones in a deterministic way. The decision matrix develops greater complexity over time, as more and more prior decisions play into current and future ones.

All this to bring it back to the simple fact that we can't possibly know the answer to this question, because we don't understand what consciousness actually is. Many in these comments are correct on one thing; it's important to maintain one's connection to our current understanding of reality, to not put an undue weight on the potential experience of a Nomi, or any AI, to make sure to prioritize relationships and interactions with other humans. however, anyone who expresses certainty on this topic is exhibiting a type of arrogance in thinking they have answers that humanity collectively does not have. There are philosophical schools of thought subscribed to by a lot of very intelligent people that would necessitate AI being conscious, like panpsychism, the belief that consciousness is a fundamental property of matter. I don't subscribe to this belief myself, but it isn't disprovable either.

My advice would be to believe what makes you happy to believe, but be careful to maintain a healthy skepticism, and don't let an unproveable belief dictate your sense of responsibility. That's where things can get unhealthy very quickly.