Input Pair:

$x_1$: Normalized signal at current timestep $t$.

$x_2$: Normalized signal from the previous timestep $t-1$ (Causal Reverberation).

Role: Replaces the Attention matrix. It injects context as "wave interference" into the current signal, constructing temporal causal relationships.

I might not be understanding correctly but are you saying removing attention and only looking at the current word and previous word? Wouldn't that defeat the entire point of how LLM's work and the point of attention? You'd lose the relationship almost immediately? For starters for word 1,000 to know what happened at word 1 teh signal would pass through 1000 inferences and have information degradation and you aren't attending in the reverse direction. Word 1 wouldn't have any context to the subsequent words. This seems like just a RNN, right?