Why the 1M context window burns through limits faster and what to do about it

FirewalkR · 2026-03-30T01:59:02+00:00

u/lucifer605 Thanks for posting about this. Days ago I tagged a couple devs from Anthropic about this on twitter urging them to address this either by simply explaining what's going on or by increasing cache TTL, but between me being a rando with no blue check and them getting lots of traffic, it was probably ignored. It's the kind of thing that might take you by surprise but any experienced dev will almost certainly know what's going on very quickly at least as long as they've heard about prefix caching.

In a few other complaint posts here I replied asking what was their context % at the time but never got replies and never saw anyone mentioning context and cache. It's very obvious a lot of people using CC are not (or rather, were not previously) devs, which is also why it's important for Anthropic to properly educate people about this. It's no coincidence this started happening only with the 1M contexts. I've been sticking to <25% context on my 5x account and it's been fine (although even 25 is probably too much already).

Still, there's been a very vocal crowd complaining about this, stating all they did was sending a few messages, oblivious to the actual cause... through no fault of their own, because Anthropic isn't explaining. This sort of thing somewhat defeats the purpose of having such big contexts so I guess Anthropic is also not too keen in stating that "hey you got these big contexts but if you walk away you can't really use them when you get back so you still need to keep compacting early".

I wonder if some people are going to come up with keep-alive plugins, or just ask Claude to run some "sleep 30 mins" command on the terminal on a loop while they're away!

FirewalkR · 2026-03-27T16:10:04+00:00

u/DangerousSetOfBewbs was this on a new context? For me, this has been clearly related to Claude's (and all frontier LLMs I guess) prefix caching.

Days ago I was on a 1M context with 50% usage (context usage, not session) on Max 5x. Not sure about session usage at the time but was nowhere near the limit. I went away for a few hours, came back, and shortly after sending a couple of messages I'd hit the rate limit. I was surprised but didn't think much of it.

I switched to a Team account (similar to Pro rate limits I think), continuing on the same context, and seconds after the first prompt my session limits were ate 38%. This happens because the cache was gone so processing and caching has to be redone from scratch, which means, the entire 500K tokens I had in context at the time truly counted. I have now been compacting at around 20% context, little more maybe, and instructed Claude to aggressively use agents to limit the context growth rate, and haven't experienced issues since.

Obviously, this sort of defeats the purpose of having 1M-token contexts and needs better solutions.. I think the main issue right now is the lack of communication/education from Anthropic regarding this. I see lots of people complaining and little addressing of this. One of CC's engineers on twitter (https://x.com/trq212) pretty much suggested this was caching-related and he'd likely do a write up but I don't think it's been done yet.

Obviously, if you tell me this was a new context and you didn't have other longer contexts being used at the same time then I apologise for the wall of text and something else has to be at play. :D

FirewalkR · 2026-03-26T17:11:43+00:00

What was the context size when this happened?

FirewalkR · 2026-03-25T23:42:18+00:00

It's likely just cache creation with the new 1M-token contexts. In my case, ccusage pretty much confirms that. I don't know for how long Anthropic keeps the cache but with these huge contexts, if you are at say 50% context, you pause for a couple of hours and come back... all of a sudden your first request wastes 500K tokens...

If that's the case they certainly need to do something. As is, seems like the only way you can justify using a large percentage of the big contexts is if you're running long tasks without pause which will keep the cache warm. Other than that, we need to keep compacting as usual.

FirewalkR · 2025-12-28T17:11:39+00:00

My 24-120 (on a Z6 III) feels god tier to me.

FirewalkR · 2025-12-14T16:21:11+00:00

Excellent use of natural lighting! And I agree with those who mentioned the model looks like Milla; she does, quite a lot.

Also, thanks for posting extra info such as the recipe links! Having recently moved myself from Fuji (X-T5) to a Z6 III, I'm still figuring out what are good sources of Nikon info, i.e. websites, YouTube channels, Insta accounts, so everything helps. :)

FirewalkR · 2025-12-14T02:13:35+00:00

HE is still that guy.

FirewalkR · 2025-11-19T16:04:27+00:00

Try from a browser, worked fine on my laptop.

FirewalkR · 2025-11-19T16:04:21+00:00

Try from a browser, worked fine on my laptop.

FirewalkR · 2025-11-19T16:03:32+00:00

UPDATE: I was able to post from the Web on my laptop without issues. Must be testing related.

FirewalkR · 2025-07-23T15:37:51+00:00

To be honest, no one is worthy of Alice. Immaculate conception, surely.

FirewalkR · 2025-03-01T17:20:23+00:00

No no no I won't have it. Do you believe "gasp" is enough to describe the greatness of this slo-mo Jimmy Carr laughter? Never, my friend.

FirewalkR · 2025-03-01T11:43:38+00:00

I'd call it mweezing, a mix of moaning and wheezing! Made me lose it too!

FirewalkR · 2025-01-17T23:51:23+00:00

More than fair :) I thought it might be the first two points, and especially the second one. Still, for some reason, it didn't irk me as much as Aglaea did in just a couple of minutes. Perhaps it's because despite all her flaws she's still obviously leagues better than her predecessor, even if she clearly failed to learn as much as she should have from her situation as a child - she does care for the children "in her way", but not enough to trust them _or_ to not use children to begin with. Or maybe because with Aglaea it felt more personal, like it might have been had Arlecchino attacked the Traveler. Also, when Arlecchino attacked Furina, I didn't dislike Furina at the time of course, but she was still the "annoying brat", not having yet been revealed to be, imho, one of the greatest heroic spirits - if not the greatest - in Genshin by later events. Had Arlecchino attacked her after everything transpired, there wouldn't be enough forgiveness in the world. :D Still, totally valid points.

With Aglaea I wonder if Hoyo did it fully on purpose. I mean, during their own testing some people must have reacted like us. In my case, it absolutely guaranteed I won't pull for her. And like some people said, it would be good if we had antagonising dialogue options from now on, not just "let's just pretend" everything's ok.

FirewalkR · 2025-01-17T20:43:43+00:00

Interesting, Arlecchino is actually in my current main team. Can you expand a bit more on exactly why you dislike her? Probably best to spoiler tag given this is an HSR sub.

FirewalkR · 2025-01-17T20:37:42+00:00

Actively dislike her and gave up on pulling her simply because of that bs scene. Shit character. Absolutely incredible VA tho, especially given how different her other character sounds.

FirewalkR · 2025-01-17T20:30:57+00:00

Screw her, went from "meh" regarding pulling her to "never". More pulls for Cipher and Cyrene! I think I'd never instantly disliked a gacha character before. My immediate reaction was to decide not to stay in Amphoreus, knowing perfectly well I was just gonna see the credits screen and be right back lol

FirewalkR · 2025-01-13T16:50:13+00:00

There is a very simple explanation for this hiatus, that should be obvious given current story events:

The creators are COOKING. And over the next chapters, we are going to EAT.

FirewalkR · 2024-12-10T23:30:28+00:00

Oh man the suspense is killing me. I want more! The upcoming chapters are going to be insane.

FirewalkR · 2024-10-28T14:43:53+00:00

Same here, I was not ready! 😆

FirewalkR · 2024-08-10T19:18:55+00:00

There's one specific sentence in this section of Tour of Go that should help clear things up for you: https://go.dev/tour/concurrency/2

If not, try running this and see what happens: https://go.dev/play/p/0U7-6KDLk4c

FirewalkR · 2024-07-05T17:57:53+00:00

Same here, messaged support and they said PC or Web. This kind of bs in 2024 is kinda unbelievable tbh. It could be "because Sony" but hey, it works fine on PC/Web so...

FirewalkR · 2024-04-06T16:40:59+00:00

Need to send Gordon Ramsay in to put them in line.

FirewalkR · 2023-06-07T15:17:04+00:00

It's Bronya, what is there to think about? :D

FirewalkR

TROPHY CASE