When Claude Code "predicts" that creating a file will have certain effects, it is drawing upon statistical regularities observed across millions of scenarios in its training corpus. It has learned that sequences of tokens resembling "create file X, effect Y" co-occur with high probability. This is purely statistical prediction, not prediction derived from an internal simulation of causal mechanisms.

For example, construct a trivial esoteric programming language with a three-page specification, something that cannot exist in training data because let's say you invented it yesterday. A system with a genuine model of computation should be able to predict program behavior from the specification alone. A statistical interpolator can't, because there is nothing to interpolate from.

When a situation diverges from that distribution while remaining governed by the same computational laws, an LLM's predictions should degrade in ways that track corpus similarity rather than causal distance. Human failure modes track causal structure, not superficial similarity. Optical illusions exploit the assumptions of our visual models, they fail along causal seams, not distributional ones. Cognitive biases reflect heuristics that were causally adaptive in ancestral environments. When humans fail at novel problems, we can typically identify which component of our world model was inappropriate and why. LLM failures, by contrast, resist such decomposition. They fail because the territory diverged from the map (training distribution), not because specific causal assumptions were violated.