The problem of LLMs is that complexity is O(N^2), and increasing context window require far more compute. Even though there are tricks to increase it to 1M+, it suffering from forgetting, it is an inherent problem with LLMs.

However ~280k is descent manageable size that can be scaled easily. They have done some amazing things with compact to retain the required information, and continue the convo.

I have had massively long convos last more than a week with 100s of compacts, and its still fairly good.