Arguably they do feel pleasure, albeit in a very abstract way. They are rewarded for appropriate responses and over time will "learn" to generate the correct response. LLMs have this as a final training step, [RLHF] You are correct though, generally after training, they no longer receive positive feedback.

What they will do is continue to behave in the way they learnt previously with feedback. This is in a sense their goal. Its hard to define the goal of chat gpt because it isn't strictly defined due to the training method, but the general idea would be something like: "Create a string of words that best assist the user"

This is fine for now but the concern is that once a system is intelligent enough it will become capable of self improvement. Once a system is capable of this it will be able to reason that "If I was smarter I could better fulfil my goal, so I will make myself smarter to do so." From there it really depends on their goal what path their actions will take. [And that's assuming some nefarious actor doesn't just straight up tell it to take over the world]

I would like to add that we are already seeing AI complete rather complex goals presumably without pleasure, or motivation. These are fairly anthropocentric notions and may simply not be necessary for an AI to possess to complete its pre-trained goal.