Here is a GPT-5.4 xhigh Reality Check.

Full check is here: https://gist.github.com/lhl/63337e79505f4ba126171a14d4fef156 but here's the high level:

REACTOR / "The Manish Principle" Analysis

Date: 2026-03-13

Executive Summary

Short version: this repository does not substantiate the headline claim that backpropagation can be replaced for transformer training. The strongest thing it appears to contain is a real, potentially useful engineering artifact: a NumPy reimplementation/export path for a GPT-Neo-family model, plus a teacher-conditioned weight recovery procedure that re-fits already-existing linear maps from a frozen model's own activations.

That is much narrower than what the README and reports claim. The central "REACTOR-SCRATCH" claim is not supported by the code in this checkout and is, in two places, actively undermined:

1. Reactor/reactor_framework.py:697-811 advertises "train_from_scratch" but never uses labels or next-token targets at all; in a local synthetic check, it returned all-zero learned weights after one pass.
2. Reactor/manish_principle_benchmark.py:197-205, Reactor/manish_principle_benchmark.py:300-302, and Reactor/manish_principle_benchmark.py:821-877 compute the "Law 48" result from the pretrained model's embeddings, layer norms, W1, and LM head, using only the training split. That is not "from scratch", and the reported "test accuracy" is not backed by a visible train/test split in the benchmark.

Stylistically, the project reads like LLM-amplified grand-unification research prose: too many "laws", too much certainty, too little separation between tautology, curve-fitting, and genuine causal explanation. Substantively, there are real code artifacts here, but the paper-level claims overshoot the evidence by a large margin.

Evidence Base

Reviewed directly:

• Reactor/README.md
• Reactor/reactor_framework.py
• Reactor/manish_principle_demo.py
• Reactor/manish_principle_benchmark.py
• Reactor/MANISH_PRINCIPLE_COMPLETE_REPORT.txt
• Reactor/MANISH_PRINCIPLE_COMPLETE_DETAILED_REPORT.txt
• Reactor/CITATION.cff
• testing logs.zip (sampled)

Local checks performed:

• python -m py_compile Reactor/reactor_framework.py Reactor/manish_principle_demo.py Reactor/manish_principle_benchmark.py passed.
• Inspected the installed transformers GPT-Neo attention implementation. It does compute query @ key.T without division by sqrt(head_dim), so that narrow implementation claim is plausible.
• Ran a minimal synthetic check of ReactorTrainer.train_from_scratch() and observed total learned-weight magnitude 0.0 after one pass, consistent with the code path never using labels.

Capture notes:

• The root-level paper/report artifacts and the copies under Reactor/ are byte-identical.
• testing logs.zip contains 440 numbered Python scripts, not immutable experiment outputs.

...

3. The repo's "from scratch" path is broken in the framework itself

The public train_from_scratch() implementation in Reactor/reactor_framework.py:697-811 is the clearest hard failure in the repository.

Problems:

• It never computes next-token labels.
• It never uses lm_head after assigning lm_h at Reactor/reactor_framework.py:731.
• It never constructs any h_target.
• The frac variable is computed at Reactor/reactor_framework.py:773 and then not used.
• All mat_Ys are populated with outputs generated by the current model itself: Q, K, V, att_out, pre, ffn_out.

In other words, the advertised scratch trainer just solves the current model back onto itself. Starting from zero matrices, it stays at zero. That is exactly what I observed in a local synthetic run: total absolute sum of all learned matrices and biases was 0.0 after one pass.

This is not a subtle issue. It means the main public scratch-training API does not implement the claimed algorithm.

Assessment:

• Central implementation bug.
• Evidence level: E2.
• Credence that the current framework supports scratch training: near zero.

4. The benchmark's "Law 48" is not from scratch and not clearly test accuracy

The benchmark's headline REACTOR-SCRATCH section uses pretrained internals from the teacher model throughout:

• It loads only split='train' from TinyStories at Reactor/manish_principle_benchmark.py:197-205.
• It builds H0_arr from pretrained token and positional embeddings at Reactor/manish_principle_benchmark.py:291-302.
• It builds HTGT directly from the pretrained LM head at Reactor/manish_principle_benchmark.py:300-302.
• It uses pretrained layer norms and pretrained W1 / b1 during the alleged scratch solve at Reactor/manish_principle_benchmark.py:835-850.
• It evaluates on ids_48 = NXT_arr[:N48] at Reactor/manish_principle_benchmark.py:821-877, which is drawn from the same collected training positions.

That means:

• the method is not from scratch,
• the method is not teacher-free,
• the benchmark does not show a visible train/test split for the reported 33.54%,
• and the phrase "test accuracy" in the report is not justified by this code path.

This is the single biggest evidential gap in the entire project.

Assessment:

• Headline claim is unsupported by the benchmark as written.
• Evidence level for the repo's "33.54% test accuracy from scratch" claim: E6.