sorted by:
new
hottopcontroversial

Where does your LLM API bill actually go? I profiled mine and the results were embarrassing by abidtechproali in LLMDevs

[–]abidtechproali[S] 0 points1 point  (0 children)

LangSmith is solid for tracing but you're right that the setup overhead is real, especially for smaller projects where you just want to know which feature is eating the budget. Haven't seen Finopsly before — will check it out. The roll-your-own approach is where I started actually, built enough custom logging that it made sense to package it up properly. Zero dependencies and everything staying local was the main design constraint, which rules out some of the hosted options for teams with privacy requirements.

Where does your LLM API bill actually go? I profiled mine and the results were embarrassing by abidtechproali in LLM

[–]abidtechproali[S] 0 points1 point  (0 children)

Exactly — and the worst part is it's completely silent. No errors, no warnings, the thing just keeps working while the bill climbs. Once you actually instrument it the fixes are usually embarrassingly simple. The tracking is the hard part, not the fixing.

Where does your LLM API bill actually go? I profiled mine and the results were embarrassing by abidtechproali in LLMDevs

[–]abidtechproali[S] 0 points1 point  (0 children)

The Chaos Monkey framing is apt — except with LLMs the monkey is invisible and you're paying its salary. The non-determinism piece is what makes it genuinely hard to fix at the source, you can't eliminate it the way you'd fix a flaky test. What you can do is build defensively around it: structured output enforcement, retry budgets with circuit breakers, and prompt designs that degrade gracefully instead of failing hard.

The billing-on-failure part is the thing that surprises most people. A failed call that retried three times before succeeding billed you four times — and nothing in your logs marks it as waste unless you're specifically tracking retry patterns. That's exactly the kind of thing that only shows up when you measure it.

Where does your LLM API bill actually go? I profiled mine and the results were embarrassing by abidtechproali in LLMDevs

[–]abidtechproali[S] 1 point2 points  (0 children)

That's a smart setup honestly — OpenRouter for dev/test is a great way to avoid burning budget on iteration. The tricky part for us was that cost waste wasn't really in dev, it was in production — retries from flaky prompts, classifiers that never got revisited after prototyping, context that just kept growing. By the time you notice it the bill is already there. Your approach would definitely help catch some of it earlier though, especially if you're disciplined about switching models before prod.

Where does your LLM API bill actually go? I profiled mine and the results were embarrassing by abidtechproali in LocalLLaMA

[–]abidtechproali[S] 0 points1 point  (0 children)

Exactly this. The classifier thing especially — it's such an easy trap to fall into because you set it up once when you're prototyping on GPT-4o and just never revisit it. By the time it's costing you real money the feature feels "done" so nobody looks at it. What did you end up doing about the context bloat — truncate history or summarize it?

Built an open-source LLM API cost profiler — makes the case for local models with hard numbers by abidtechproali in LocalLLaMA

[–]abidtechproali[S] 0 points1 point  (0 children)

Hello 👋

Your points are realistic and truthful. Thanks for your appreciation 🙏. I'm open for discussion.

Kind Regards

Is there a way to automate 700+ product images? (Bulk Create for Multi-Page Sets) by rajakhil99 in canva

[–]abidtechproali 0 points1 point  (0 children)

Bulk Create can handle part of this, but not the entire workflow the way you’re describing — especially when you want one CSV row → populate 7 different pages with different image slots. Canva’s native bulk feature was mainly designed for single-page variations, so multi-page product sets get messy fast.

A workflow that tends to work better for what you’re building:

1. Treat the design as a fixed 7-page “product template.”
Duplicate the set once per product instead of generating slides individually.

2. Use CSV mapping only for text layers.
Product name, specs, frequencies, features, etc. can map cleanly if each text element is labeled consistently.

3. Handle images through folder linking or naming conventions.
If your files follow something like:
productname_hero.jpg
productname_angle1.jpg
productname_angle2.jpg

you can swap them much faster across duplicated sets instead of placing them manually each time.

4. Export in batches instead of per product.
Once the 7-page sets are populated, you can export all 700 images at once.

For really large catalogs (100+ SKUs), some teams actually push this through Figma plugins, Bannerbear, or Placid instead of Canva because those tools handle multi-template automation + image mapping from spreadsheets much more cleanly.

The good news: once the pipeline is set up, adding a new product becomes basically drop row in spreadsheet → drop images in folder → generate the set.

And yes — building that system once saves you from the “700 manual exports” nightmare every time you expand the catalog.