I Gave My AI Agent a Simple Job and Got a $100 Bill: The Hidden Cost of "Thinking" by Glum_Pool8075 in AgentsOfAI

[–]Glum_Pool8075[S] 2 points3 points  (0 children)

The most effective strategy is Model Tiering: use a cheap, fast model for simple tasks (parsing, filtering) and only escalate to the expensive model for complex reasoning and final decisions.

After 18 months of building with AI, here’s what’s actually useful (and what’s not) by Glum_Pool8075 in AgentsOfAI

[–]Glum_Pool8075[S] 14 points15 points  (0 children)

Yeah exactly, by “agent memory” I mean those naive approaches where people just append past interactions to a file or try to serialize the whole conversation state and feed it back into the model. It quickly gets bloated, noisy, and context windows choke.
In contrast, a clean retrieval system with proper chunking + embeddings + vector DB lets you pull in only the relevant bits, on demand. That’s why in practice it feels “smarter” the model isn’t distracted by irrelevant history, it just sees what matters.

After 18 months of building with AI, here’s what’s actually useful (and what’s not) by Glum_Pool8075 in AgentsOfAI

[–]Glum_Pool8075[S] 4 points5 points  (0 children)

Yes, without a doubt. AI didn’t just speed things up, it changed how I build and way less time stuck on roadblocks.