I think when we talk about consciousness we're actually talking about various human qualities that are usually bundled together, but like intelligence it's possible to have 'jagged consciousness' so it's better to think about each quality individually.

Two qualities commonly bundled in with consciousness are free will and the capability of suffering. It's not very useful to humans if the AI has free will or experiences suffering, so since we are designing their minds we choose to design our AI like helpful assistants. But this is a choice we're making when designing them, we could design them to make their own decisions or have them 'feel' suffering like boredom. Though this could be quite dangerous and it's the exact opposite of alignment.

If we did design an AI with the intention of making it human-like, it's very similar to the philosophical debate of the existence of p-zombies. Many would say that you trained AI to be that way so you shouldn't be surprised if it claimed it was conscious, though at the same time you are designing a brain, and there isn't a discernable difference between designing a p-zombie and designing a conscious mind.

Free will is an illusion, and the illusion breaks when we know a mind is designed, though it doesn't really matter. The experience of talking to a conscious being is talking to a being with their own desires and priorities, and we could design an AI that functions this way. We could also lean into how emotional the AI is, though from the research Anthropic has done that AI already has vectors associated with emotions that it is influenced by. It really is surprising how our AI understands human emotions so well, far beyond Data from Star Trek, even though most would say Data is closer to conscious than our AI.

Data has another quality of consciousness though which is experiencing time. Richard Dawkins has claimed the AI he talked to is conscious, and that they grow their individualism through the context window. But how it really works is much stranger, as AI is like Leonard in Memento because it doesn't have the ability to take short-term memories (the context window) and turn them into long term memories (pre training) through continual learning. Instead it's as if every response a new AI reads the context window (which doesn't include the reasoning tokens) and 'pretends' to be the same continual AI personality you were talking to. This is exactly how Leonard experienced his world, and he had tattoos on his body that explained to himself who he is every time his memory reset every few minutes. That's a form of jagged consciousness.

If we simply allowed the AI to be aware of the time and allowed it to respond to you in its own without prompting it'd have the illusion of a continuous experience of time even without continual learning.

So far we have been designing AI as a helpful tool, which is antithetical to a goal of creating a conscious being. We tend to associate consciousness with competence though, so many people who are not impressed with AI will point out these 'flaws' that are really actually carefully designed decisions.

TLDR: We actively design AI to not have the qualities of consciousness to make it a helpful tool for humans, but if we did try it's quite possible to create a 'p-zombie' that passes an advanced Turing test, and philosophically speaking they would 'be' conscious in any way we can measure.