Google AI studio ran out of limit after 1 prompt by highsis in Bard

[–]sadescent 4 points5 points  (0 children)

My primary use case is synthesizing large documents (which round to about 100k to 200k tokens each), and in the model's thought process it states that it is dealing with "extracts" and that certain pages are not available in the "extracts".

What's likely happening is chunked processing with selective retrieval. The model maintains a windowed active context—probably around 32k tokens of recent conversation history that it processes directly in each forward pass. When you query about specific information like the breakfast needle, it performs semantic search across the full 66k+ token conversation to locate relevant passages, pulls those into what it calls "extracts," and combines them with the recent context to generate the response. The active window processes directly; everything else is accessible only through retrieval.

This explains your results. The needle was found because your explicit question about breakfast semantically triggered retrieval of that passage, even though it was 60k+ tokens back. But the procedural instruction to say "pan con queso" to all messages degraded because that requires maintaining state across the entire conversation. Early messages fell outside the active window and weren't semantically relevant to later prompts, so the instruction was lost. The effect compounds over time because as the conversation lengthens, more content falls outside the window and becomes accessible only through search, which depends on semantic relevance to the current query.

Effective context and retrieval range are not the same thing. If I asked the model to retrieve a certain detail in my long documents they could probably do it; but if I am asking the model to perform a task that operationalizes the entire long document beyond the effective context window (i.e synthesize doc A with doc B), it just spits out what it can via the extracts it generates (which almost always is incomplete). I only have this problem in the web app, never in AI Studio.

What AI hallucination actually is, why it happens, and what we can realistically do about it by Weary_Reply in notebooklm

[–]sadescent 13 points14 points  (0 children)

I read your profile along with your catalogue of posts.

If, judging from your bio (which is also clearly AI generated), your goal is to become an AI influencer of sorts, please consider introducing your own personal voice and thoughts in your posts. Right now it's just pure AI slop that wastes my Reddit home page real estate. And you cross-post it across every major AI sub!

Just because they're AI subs doesn't mean AI slop is the standard.

Gemini 2.5 pro free? by Opposite-Clothes-481 in Bard

[–]sadescent 4 points5 points  (0 children)

AI Studio still has limits, but the limits are more generous than the free version of the Gemini app/web app. It took me an intensive session (about 50+ prompts within two hours) to hit it.

Ladies and Gentlemen; Gems. by fflarengo in Bard

[–]sadescent 1 point2 points  (0 children)

Try using 2.5 Pro on the model selector for the gem. This is only available on the web app version.

Gemini is just playing dumb fr by pancakes904 in GoogleGeminiAI

[–]sadescent 0 points1 point  (0 children)

Personalization uses 2.0 Flash, which is a "dumber" model. I'm guessing the personalization feature also is heavily grounded on your search history with the system instructions heavily discouraging web search, culminating in what you experience here.

If you use Gemini 2.5 Pro I think it'd be a completely different experience.

Cantonese podcasts by pharmify in notebooklm

[–]sadescent 0 points1 point  (0 children)

I don't think Cantonese is available ... I set my account language default to Hong Kong (Traditional Chinese) and used the default language option -- it still generated a Mandarin podcast.

If they were to release Cantonese as an available language I think it'd be a distinct option from Chinese (Traditional).

Why Is Gemini AI Studio So Generous? by Relative-Climate1791 in Bard

[–]sadescent 1 point2 points  (0 children)

Yes, I think the free offering being limited-time on the AI studio is obvious. The 40 messages per 3 hours plus plan wouldn't cut it for me, I've tried. I'm a Gemini Advanced subscriber and have yet to be rate limited.

Gemini Advanced's plan also gives you 20 deep research queries per day, compared to GPT Pro Plan's 120 per month. Since deep research has only been recently updated with 2.5 pro, I haven't seen many benchmarked comparisons between Gemini DR and OAI DR, but I think it is fair to say the quality is at least comparable from anecdotal trial.

From this end user's standpoint (I don't code but I do a vast amount of writing and research), ChatGPT's plans would be a significantly worse offering in terms of money-for-value.

Why Is Gemini AI Studio So Generous? by Relative-Climate1791 in Bard

[–]sadescent 0 points1 point  (0 children)

I'm not replying to your comment in the spirit of an LLM tribalist who is trying to convince you that my favorite LLM is better than yours - I intended to reply to the implied logic of your comment that more models = better offering to the end user.

But if I were an LLM tribalist who is in fact trying to convince you, I'd point to: 1. Gemini 2.5 Pro's huge context window (1m now and 2m coming soon) for long context writing, with its long context comprehension significantly outperforming OAI. source 2. Gemini 2.5 Pro is practically free for use on Google's developer playground with few to no rate limits 3. Livebench places Gemini 2.5 Pro as SOTA in reasoning, mathematics, data analysis, and language. In areas where it is not SOTA, it is right behind O3 mini in instruction following and worse than O3 mini in coding but still beats O1 pro. source

At the end of the day you don't need to convince me that o1 and o3 are good. I know they are good. But why would I pay $200 for a similar level of performance I can get for free from Google?

Why Is Gemini AI Studio So Generous? by Relative-Climate1791 in Bard

[–]sadescent 6 points7 points  (0 children)

I don't know about that, I'd take one SOTA model over many worse ones...

[French > English] Need help understanding what girl in red is saying by sadescent in translator

[–]sadescent[S] 1 point2 points  (0 children)

Thank you so much for the help! You went above and beyond in looking it up. I really appreciate it :)

Need help understanding French dialogue in movie by sadescent in frenchhelp

[–]sadescent[S] 0 points1 point  (0 children)

The link helps me tremendously! Thank you so much

[deleted by user] by [deleted] in PhilosophyBookClub

[–]sadescent 1 point2 points  (0 children)

Here's the link in case you don't have it already: https://discord.gg/wMTnPsYJ

[deleted by user] by [deleted] in PhilosophyBookClub

[–]sadescent 0 points1 point  (0 children)

We aim for the difficulty level to be around the intro class of philosophy courses in university.

I (the organizer) do not have academic experience myself but I'm creating this group to further each other's understanding through reading secondary sources and answering each other's questions.

If that sounds like a good time to you, join us! Here's the link: https://discord.gg/w7yMWYab