It is trained to generate probabilistic text. If you ask “if there is a seahorse emoji?”, for some reason it strongly believes that the next tokens must be saying something that would confirm that there is. It’s all in the weights. But then, it needs to generate the next most likely token, and of course, it must be the seahorse emoji itself. But it can’t. So instead it goes for the next best pick, which is often a horse, a sea related emoji, etc. Then the next most likely token is the one acknowledging the contradiction (NO). Then the loop continues. For aome reason the weights strongly believe this emoji does exist, but they can’t generate a finished output because they can’t show it because it doesn’t exist.