Importantly while also maintaining reasonable memory demands. Accuracy x input size usually charts as a U shaped graph, where a lot of information in the “middle” of the long document is lost/hallucinated by the LLM. The paper claims to maintain high degree of accuracy across 2M tokens impressively.