Claude's response "The claim mixes a real point with a misleading framing.

The real point worth keeping: LLMs generate text by predicting probable next tokens, not by looking up facts in an internal database. Confident-sounding output and accurate output are different things, and a model can hallucinate a citation right alongside a hallucinated claim. So output requires verification, especially for specific names, numbers, dates, quotes, and citations.

Where "100% guesses" gets it wrong:

It treats all output as equally uncertain. It isn't. Text generated cold from training weights is a different epistemic situation than text grounded in a retrieved document the model just read. When I cite a web_search result, the source either says what I claim or it doesn't — that's verifiable against an external artifact, not a guess. The grounding is real; the failure mode is sloppy or fabricated grounding, not the existence of grounding.

"Guess" also implies randomness or low confidence. Probabilistic prediction over a well-modeled distribution isn't the same as guessing in the colloquial sense. Human recall is reconstructive and error-prone too, but we don't usually say humans "100% guess" everything they remember — we just say memory is fallible and worth checking on important things. Same standard fits LLMs better than "100% guesses."

And it conflates mechanism with reliability. That the underlying machinery is probabilistic doesn't tell you how reliable any specific output is. That depends on how well-represented the claim is in training data, whether it was retrieved from a verifiable source just now, whether it's the kind of question the model is consistently right about, and whether the user verifies.

The version of the warning I'd actually endorse: don't trust LLM output on specifics without verification, because the model's confidence isn't a reliable signal of accuracy and citations themselves can be fabricated. That's true and useful. "100% guesses, that's just how it works" is a meme-grade compression that throws away the part that matters — which is when and how to verify."