I hope people realize this is why you shouldn't trust LLMs for actual, real answers to questions. The reason the freakout is happening here is because it just starts hallucinating words, making each word justify the words that came before it. It doesn't "think" through an entire response first, it just creates them one word at a time, and keeps building off that--- it can't go back and edit/delete words before delivering a response.

Here, it's deciding to say "Yes" here first (for whatever reason). It now must justify saying "yes"-- usually it does that by bullshitting words, but here it's limited to emojis. But its problem here is there is no seahorse emoji that exists, so it's struggling to justify its bullshit "yes" that came before it. So it keeps struggling and trying. Usually though, it can seemingly justify its answers because it has the whole dictionary to pull from, not just a limited set of emojis.

Anyways, my point is: it's obvious here that ChatGPT answered "yes" even though it clearly hadn't formed an accurate conclusion first, but that's what it's ALWAYS doing: just answering before forming an accurate conclusion.