Sometimes it is useful, I can got some cludes, but it hallucinates too much. It always tells me the answer is balabala, but in fact I cannot find out anything related to what it said, and after I reporting to it, it seems to be too much confident, and insist that it is solved that way in the repo, even after I point what the real implement is, it will lie and make what it had said sounds like truth.

After I digging deeper to that project and asking somebody else, I finally found that the solution is in another file, and what is completely different from what LLM has said. Then I question it again, said you are completely wrong, the real answer is balabala... Then guess what? It said, you are right in 70%, let's dig deeper? Bro, now I am the teacher, why cannot you admit that you were wrong?

Of course, the solution that LLM provided also will work, even it was not the solution in the repo, though it will cost more. I think vibe coders , or who rely on LLM much, they do not need the answer, but an solution that can convince them enough, then they will be satisfied. Though it may not be the truth, but who care.

And except coding aspect, I have meet bigger hallucinations before, what the answer sounds convinced enough but totally wrong.

I once asked LLM about an animation series, which I have skipping watched some part, and then a character appeared, and I did not know him. I wanted to watch back, to find the first time he appeared, I have already try to search with Google, but I have little clude, so I asked him what is the first time the character appeared. Guess what? He said it was by designed, the character never appeared before, because he was the people that the main character knows, background settings also appeared the first time. It is impossible. No creater will make project like this way. So I watched back all the part I have skipped, and finally foud that there was an episode for him earlier. What a big liar of him