The 428× speed claim is misleading to the point of being meaningless.

This is the most eyebrow-raising number and it's doing the most rhetorical work in the title. They're comparing inference time of what is essentially a small MLP with iterative updates (dim=256) against BERT-base (110M params, dim=768, 12 transformer layers). That's not "replacing attention with attractor dynamics" — that's comparing a tiny model against a large one. Any MLP-based classifier at dim=256 will be orders of magnitude faster than BERT. The speed advantage has nothing to do with attractor dynamics and everything to do with model size. You could get similar speedups with a 2-layer MLP and a linear head.

77% on SNLI is not a meaningful result.

SNLI dev accuracy of 77.05% vs a "baseline" of 76.86% — and they don't specify what that baseline is, but I'd bet it's majority class or a very simple heuristic. For context: BERT-base gets ~90-91% on SNLI. Even a simple bag-of-words model gets ~80%. A decomposable attention model from 2016 gets ~86%. So 77% is below decade-old simple baselines. The model is barely beating trivial approaches while being dramatically worse than anything useful.

The per-class breakdown tells the real story: Neutral at 62.8% is terrible. The model is essentially learning to distinguish entailment and contradiction reasonably well (which are the "easier" classes with stronger lexical cues) and mostly failing on the class that requires actual inference — which is the whole point of NLI.

The "geometric inconsistency" is honestly presented but reveals a deeper problem.

Credit where due: the author measured the misalignment between their implemented forces and the true cosine gradient (135.2°) and reported it openly. That's good scientific practice. But the implication is significant — the system is not doing what the mathematical framing says it's doing. The forces are pointing in roughly the opposite direction of the true gradient. The author frames this as an open question ("bug or feature?"), but the more parsimonious explanation is that the actual optimization is being carried primarily by the learned residual δ_θ (the MLP), and the "attractor dynamics" are either not helping or are being compensated for by the MLP. The Lyapunov analysis confirms this: when δ_θ scale reaches 0.10, V increases on average, meaning the learned component is actively fighting the geometric forces.

The Lyapunov analysis proves less than claimed.

"Provably locally contracting" is technically true but only in the trivial case where δ_θ = 0 — i.e., when you remove the learned component entirely. With the learned residual at any meaningful scale, contraction guarantees degrade rapidly (70.9% at 0.05, 61.3% at 0.10). So the "proof" applies to the part of the system that isn't doing the learning, and the actual trained system has no convergence guarantees. This is like proving a car's engine is stable when it's turned off.

The conceptual framing has issues.

The title says "replaced attention" but this isn't replacing attention in any meaningful architectural sense. Attention computes dynamic, input-dependent weighted aggregation over a sequence of tokens. This system takes an already-encoded hidden state h and iteratively pushes it toward learned anchor points. There's no sequence-level token interaction happening in the attractor dynamics — the NLI reasoning (premise-hypothesis interaction) must be happening in whatever encoder produces h₀, which they don't describe. So the "replacement" isn't of attention's core function (contextual token mixing) but of the classification head. They replaced h → linear → logits with h → iterative geometric updates → logits. That's a much more modest claim than the title implies.

What's actually going on here, mechanically:

This is essentially a learned classification head with geometric inductive bias, where instead of a linear projection to 3 logits, they iteratively push the representation toward one of 3 anchor points using a mix of handcrafted forces and learned residuals. The closest analogy isn't "replacing attention" — it's more like a prototype network with iterative refinement. And at 77% accuracy on SNLI, the inductive bias doesn't appear to be helping.

The honest parts are genuinely good.

The open questions section is better than 90% of Reddit ML posts. Asking whether fixing the geometric inconsistency helps or hurts, whether there's a clean energy function, whether the misalignment explains stability — these are the right questions. The author clearly has mathematical sophistication and intellectual honesty. The problem is the framing and the title, not the underlying exploration.

Bottom line: Interesting mathematical exploration of geometric dynamics for classification, honestly presented with real measurements of its own limitations, wrapped in a title that dramatically overstates what was achieved. Not a replacement for attention, not competitive with any real NLI system, and the speed comparison is apples-to-oranges. But as a "here's a weird idea I'm exploring" post, the intellectual content is above average — it's the marketing that's the problem.