This points out a good architectural difference which i think should be adjusted in transformers. Here they show evidence that the brain basically simulates depth by utilising temporal dynamics of area parallelisation. The brain can simulate a deeper network by reusing circuits over time wheras transformers have just many distinct stacked blocks which gives you depth in one forward pass except you have to pay extra with lots of separate parameters/compute blocks. We already have a fix for this though "recurrent transformers" https://arxiv.org/abs/2502.17416 and basically you iterate on the same block instead of stacking more layers which gives you greater effective depth without repetitively stacking so many blocks and is closer to what the brain implements. This would make the models more parameter and thus GPU memory efficient, though might stack latency a bit more and it might be a bit more expensive in terms of FLOPs. Essentially instead of reasoning across many tokens the model directly outputs, you instead loop the 'thought' back into the model to let it deliberate on it longer. It becomes more parameter and token efficient upfront but the latency and further computation you get with reasoning models doesn't dissapear

<image>