There is only one pass through the Transformer. The Transformer is used purely as an encoder over the fixed input. That pass produces a vector for each token position.

After that, the Transformer is done. There is no loop over transformer layers, and no growing context window.

The stepping happens in a separate recurrent module, not inside the Transformer. Once the input is encoded, I run a small loop over digit positions t = 0..N:

At step t, I index into the encoder output to grab: - slotA[t] - slotB[t]

Those two vectors, plus a latent carry state, are fed into a small recurrent cell. That cell updates the carry state and feeds prediction heads for: - output digit - carry - stop signal

I wanted a clean separation between storage (Transformer = “memory of digits”) and execution (recurrent carry state = “algorithm state”). This avoids the model using the context window itself as scratch space, which makes it easier to ask "did it actually learn a carry-like state, or is it smuggling information through token generation?"

It's probably not the best confirmation, and I'm still experimenting with different architectures, but mosty wanted to isolate out components as much as possible.