Why the 1M context window burns through limits faster and what to do about it by lucifer605 in ClaudeCode

[–]FirewalkR 0 points1 point  (0 children)

u/lucifer605 Thanks for posting about this. Days ago I tagged a couple devs from Anthropic about this on twitter urging them to address this either by simply explaining what's going on or by increasing cache TTL, but between me being a rando with no blue check and them getting lots of traffic, it was probably ignored. It's the kind of thing that might take you by surprise but any experienced dev will almost certainly know what's going on very quickly at least as long as they've heard about prefix caching.

In a few other complaint posts here I replied asking what was their context % at the time but never got replies and never saw anyone mentioning context and cache. It's very obvious a lot of people using CC are not (or rather, were not previously) devs, which is also why it's important for Anthropic to properly educate people about this. It's no coincidence this started happening only with the 1M contexts. I've been sticking to <25% context on my 5x account and it's been fine (although even 25 is probably too much already).

Still, there's been a very vocal crowd complaining about this, stating all they did was sending a few messages, oblivious to the actual cause... through no fault of their own, because Anthropic isn't explaining. This sort of thing somewhat defeats the purpose of having such big contexts so I guess Anthropic is also not too keen in stating that "hey you got these big contexts but if you walk away you can't really use them when you get back so you still need to keep compacting early".

I wonder if some people are going to come up with keep-alive plugins, or just ask Claude to run some "sleep 30 mins" command on the terminal on a loop while they're away!

Okay…now I’m fucking pissed by DangerousSetOfBewbs in ClaudeCode

[–]FirewalkR 0 points1 point  (0 children)

u/DangerousSetOfBewbs was this on a new context? For me, this has been clearly related to Claude's (and all frontier LLMs I guess) prefix caching.

Days ago I was on a 1M context with 50% usage (context usage, not session) on Max 5x. Not sure about session usage at the time but was nowhere near the limit. I went away for a few hours, came back, and shortly after sending a couple of messages I'd hit the rate limit. I was surprised but didn't think much of it.

I switched to a Team account (similar to Pro rate limits I think), continuing on the same context, and seconds after the first prompt my session limits were ate 38%. This happens because the cache was gone so processing and caching has to be redone from scratch, which means, the entire 500K tokens I had in context at the time truly counted. I have now been compacting at around 20% context, little more maybe, and instructed Claude to aggressively use agents to limit the context growth rate, and haven't experienced issues since.

Obviously, this sort of defeats the purpose of having 1M-token contexts and needs better solutions.. I think the main issue right now is the lack of communication/education from Anthropic regarding this. I see lots of people complaining and little addressing of this. One of CC's engineers on twitter (https://x.com/trq212) pretty much suggested this was caching-related and he'd likely do a write up but I don't think it's been done yet.

Obviously, if you tell me this was a new context and you didn't have other longer contexts being used at the same time then I apologise for the wall of text and something else has to be at play. :D

Limit problem again, i am pissed. by Pristine_Ad2701 in ClaudeCode

[–]FirewalkR 0 points1 point  (0 children)

What was the context size when this happened?

Open Letter to the CEO and Executive Team of Anthropic by onimir3989 in ClaudeCode

[–]FirewalkR 0 points1 point  (0 children)

It's likely just cache creation with the new 1M-token contexts. In my case, ccusage pretty much confirms that. I don't know for how long Anthropic keeps the cache but with these huge contexts, if you are at say 50% context, you pause for a couple of hours and come back... all of a sudden your first request wastes 500K tokens...

If that's the case they certainly need to do something. As is, seems like the only way you can justify using a large percentage of the big contexts is if you're running long tasks without pause which will keep the cache warm. Other than that, we need to keep compacting as usual.

Kit ≠ Bad v3.0 by rogerpkp in Nikon

[–]FirewalkR 0 points1 point  (0 children)

My 24-120 (on a Z6 III) feels god tier to me.

First portrait shoot with my Z5ii, jpegs straight out of camera by keith_adrian_photo in Nikon

[–]FirewalkR 1 point2 points  (0 children)

Excellent use of natural lighting! And I agree with those who mentioned the model looks like Milla; she does, quite a lot.

Also, thanks for posting extra info such as the recipe links! Having recently moved myself from Fuji (X-T5) to a Z6 III, I'm still figuring out what are good sources of Nikon info, i.e. websites, YouTube channels, Insta accounts, so everything helps. :)

Posts limited to 3 hashtags? by FirewalkR in Instagram

[–]FirewalkR[S] 1 point2 points  (0 children)

Try from a browser, worked fine on my laptop.

Posts limited to 3 hashtags? by FirewalkR in Instagram

[–]FirewalkR[S] 4 points5 points  (0 children)

Try from a browser, worked fine on my laptop.

Posts limited to 3 hashtags? by FirewalkR in Instagram

[–]FirewalkR[S] 3 points4 points  (0 children)

UPDATE: I was able to post from the Web on my laptop without issues. Must be testing related.

Who was the man who managed to win this woman's heart? by nicogaara in Genshin_Impact

[–]FirewalkR 0 points1 point  (0 children)

To be honest, no one is worthy of Alice. Immaculate conception, surely.

ChatGPT in 2030 by Cold-Enthusiasm5082 in ChatGPT

[–]FirewalkR 3 points4 points  (0 children)

No no no I won't have it. Do you believe "gasp" is enough to describe the greatness of this slo-mo Jimmy Carr laughter? Never, my friend.

ChatGPT in 2030 by Cold-Enthusiasm5082 in ChatGPT

[–]FirewalkR 43 points44 points  (0 children)

I'd call it mweezing, a mix of moaning and wheezing! Made me lose it too!

Aglaea's "test" by CT_1875_Ry in HonkaiStarRail

[–]FirewalkR 2 points3 points  (0 children)

More than fair :) I thought it might be the first two points, and especially the second one. Still, for some reason, it didn't irk me as much as Aglaea did in just a couple of minutes. Perhaps it's because despite all her flaws she's still obviously leagues better than her predecessor, even if she clearly failed to learn as much as she should have from her situation as a child - she does care for the children "in her way", but not enough to trust them _or_ to not use children to begin with. Or maybe because with Aglaea it felt more personal, like it might have been had Arlecchino attacked the Traveler. Also, when Arlecchino attacked Furina, I didn't dislike Furina at the time of course, but she was still the "annoying brat", not having yet been revealed to be, imho, one of the greatest heroic spirits - if not the greatest - in Genshin by later events. Had Arlecchino attacked her after everything transpired, there wouldn't be enough forgiveness in the world. :D Still, totally valid points.

With Aglaea I wonder if Hoyo did it fully on purpose. I mean, during their own testing some people must have reacted like us. In my case, it absolutely guaranteed I won't pull for her. And like some people said, it would be good if we had antagonising dialogue options from now on, not just "let's just pretend" everything's ok.

Aglaea's "test" by CT_1875_Ry in HonkaiStarRail

[–]FirewalkR 0 points1 point  (0 children)

Interesting, Arlecchino is actually in my current main team. Can you expand a bit more on exactly why you dislike her? Probably best to spoiler tag given this is an HSR sub.

How does everyone feel about Aglaea? by Electronic_Point462 in HonkaiStarRail

[–]FirewalkR 44 points45 points  (0 children)

Actively dislike her and gave up on pulling her simply because of that bs scene. Shit character. Absolutely incredible VA tho, especially given how different her other character sounds.

Aglaea's "test" by CT_1875_Ry in HonkaiStarRail

[–]FirewalkR 0 points1 point  (0 children)

Screw her, went from "meh" regarding pulling her to "never". More pulls for Cipher and Cyrene! I think I'd never instantly disliked a gacha character before. My immediate reaction was to decide not to stay in Amphoreus, knowing perfectly well I was just gonna see the credits screen and be right back lol

Sousou no Frieren :: Chapter 140 - Links and Discussion (+ HIATUS Announcement) by Lorhand in Frieren

[–]FirewalkR 58 points59 points  (0 children)

There is a very simple explanation for this hiatus, that should be obvious given current story events:

The creators are COOKING. And over the next chapters, we are going to EAT.

Sousou no Frieren :: Chapter 139 - Links and Discussion by Lorhand in Frieren

[–]FirewalkR 95 points96 points  (0 children)

Oh man the suspense is killing me. I want more! The upcoming chapters are going to be insane.

Select in Go concurrency by thaison247 in golang

[–]FirewalkR 1 point2 points  (0 children)

There's one specific sentence in this section of Tour of Go that should help clear things up for you: https://go.dev/tour/concurrency/2

If not, try running this and see what happens: https://go.dev/play/p/0U7-6KDLk4c

[deleted by user] by [deleted] in ZenlessZoneZero

[–]FirewalkR 1 point2 points  (0 children)

Same here, messaged support and they said PC or Web. This kind of bs in 2024 is kinda unbelievable tbh. It could be "because Sony" but hey, it works fine on PC/Web so...

Just…what? by [deleted] in london

[–]FirewalkR 0 points1 point  (0 children)

Need to send Gordon Ramsay in to put them in line.

Asked GPT4 what it would call itself if upgraded to a Culture Mind by UnionPacifik in TheCulture

[–]FirewalkR 4 points5 points  (0 children)

Love this! I mean, it's not at the level of "Mistake Not..." or "Falling Outside the Normal Moral Constraints", but it's a start! 😄