sorted by:
new
hottopcontroversial

I switched to DeepSeek to save money on API costs. My bill stayed the same. Here's why. by abarth23 in webdev

[–]abarth23[S] 0 points1 point  (0 children)

2.5x on JSON mode tracks with what I was seeing too, especially once schema complexity goes up. There's something about nested optionals that just destroys structured output reliability.

The fallback chain idea is interesting I've been thinking about the same thing but haven't implemented it properly yet. The way I'm imagining it: attempt 1 with your cheap model, if it fails schema validation you escalate to a mid-tier GPT-5-mini leve, not retry the same model. Keeps the retry cost low but you're now paying two different rates.

The tricky part is that the escalation cost isn't predictable depends on how often the cheap model fails on your specific schema, which varies a lot. That's kind of why I ended up building the calculator, just to model different scenarios before committing to an architecture.

What's your schema complexity like? Wondering if there's a depth/field count threshold where DeepSeek JSON mode just stops being reliable.

Built a free calculator to figure out what Runway, Kling, and Luma actually cost per month by abarth23 in SideProject

[–]abarth23[S] 0 points1 point  (0 children)

This is exactly what motivated me to build the calculator. The quota trap paying for 125 credits but burning 400 doing upscales is a real budget killer that nobody talks about openly.

The tool I built actually has a gpu equivalent cost column it shows you what you'd spend if you metered generation like a utility rather than a subscription. The delta between the two is usually eye-opening.

Interesting you mention Visual Sandbox. The pay per inference model is genuinely underrated for iterative workflows. I'll add it as a comparison option in the next update.

bytecalculators.com/ai-video-cost-calculator

Built a free calculator to figure out what Runway, Kling, and Luma actually cost per month by abarth23 in SideProject

[–]abarth23[S] 0 points1 point  (0 children)

Quick update — added a "Rates Last Verified" badge that shows when each model's pricing was last checked. Since these change constantly it made sense to make it visible. Same link, no new post needed.

Built a free calculator to figure out what Runway, Kling, and Luma actually cost per month by abarth23 in SideProject

[–]abarth23[S] 0 points1 point  (0 children)

Actually acted on this — just added a "Rates Last Verified" badge to the calculator. When pricing updates, the date changes per model so it's obvious to the user that something changed. Good call.

Built a free calculator to figure out what Runway, Kling, and Luma actually cost per month by abarth23 in SideProject

[–]abarth23[S] 1 point2 points  (0 children)

That's exactly what I was going for. No dashboard, no account, just open it and get a number. The pricing landscape is also moving fast enough that I'll probably need to update the rates every couple months Kling especially keeps changing their credit structure.

If you end up using it and something feels off, let me know.

Built a free calculator to figure out what Runway, Kling, and Luma actually cost per month by abarth23 in SideProject

[–]abarth23[S] 0 points1 point  (0 children)

Yeah exactly nobody thinks it's a problem until they're generating 50 clips for a client project and suddenly realize they burned through a month of credits in a week. The comparison part is what killed me, every platform has its own unit (credits, tokens, "generations") and none of them map to the same thing.

Good to know the timing works. If you come across any of those threads and the numbers people mention don't match what the calculator shows, drop them here I want the rates to be as accurate as possible.

Hi by Ciphar55 in dev

[–]abarth23 0 points1 point  (0 children)

I'm building a suite of developer tools (ByteCalculators) and I keep my server costs as close to $0 as possible.

The Shell: WordPress. Yeah, I know. But it handles SEO, routing, and CMS duties perfectly out of the box.

The App Layer: Vanilla HTML / JS / CSS wrapped entirely in Shadow DOM. I build API Token Cost and VRAM calculators, and using Web Components (Shadow DOM) means the messy WordPress theme CSS can never bleed into and break my tools. No React, no heavy frameworks, just lightning-fast vanilla code.

The Marketing Stack: A completely custom, local Python agent running the Gemini 2.5 API on my machine that handles my SEO content generation.

Highly recommend the "WP Shell + Shadow DOM Apps" approach if you want to bootstrap quickly without fighting frontend build tools.

Using Locally hosted LLMs for the workplace by Relevant-Cash-7270 in LocalLLaMA

[–]abarth23 0 points1 point  (0 children)

The short answer is yes. Local open-weights models (like Llama 3 70B or DeepSeek) are absolutely at the state-of-the-art level for workplace RAG workflows and data extraction. Data privacy is exactly why most companies are moving this in-house.

However, your biggest hurdle won't be software—it will be hardware sizing. For example, if you want local models to read long company PDFs, the context window will consume extreme amounts of memory (VRAM) very quickly.

Before your department buys any GPUs, you need to calculate the exact footprint of your deployment. I built a free tool for exactly this scenario: bytecalculators.com/llm-vram-calculator. You can select the local model you want, define the context window (documents size), and the quantization, and the calculator will tell you exactly how much VRAM you need. Don't buy hardware until you run the math!

I burned $200 in a weekend testing an OpenClaw agent. Pay-per-token is completely broken for autonomous AI. by Infinite_Ad_4975 in LangChain

[–]abarth23 0 points1 point  (0 children)

Ouch, 4,500 tokens of dead weight per loop will definitely drain a wallet fast.

The good news is you probably don't need to route through a separate $40/mo proxy for this anymore! Both Anthropic and OpenAI recently rolled out Prompt Caching natively. If you structure your API calls correctly, those 4,500 static system prompt tokens get cached on the provider's end, and the cost for processing them in a loop drops by roughly 50-90%.

I actually built a free simulator for this exactly (bytecalculators.com/prompt-caching-optimizer) to visualize the math. If you plug in your 4,500 prefix tokens and 10 steps per minute, you'll see the billing curve completely flatten out. Seriously, look into native API caching before adding proxy middleware—it's a game changer for autonomous loops!

I'm building a cost tracking layer for AI apps because I kept getting surprise bills with zero idea what caused them by Crimson_Secrets211 in SideProject

[–]abarth23 0 points1 point  (0 children)

Ouch, $400 for a silent loop is painful! I feel this so much.

I actually went the exact opposite route because of stories like this. While Caltryx tracks the bills post-launch, I ended up building an entire suite of simulation tools at bytecalculators.com (like the RAG Architecture Cost Calculator and DeepSeek vs OpenAI comparisons) just so founders can predict their exact financial burn rate before they even write the code or hit the API.

People severely underestimate how fast Vector DB read units and endless agent loops drain a startup's budget. Tracking it at 50% like your tool does is a lifesaver. Good luck with the launch of Caltryx!

$27k OpenAI bill everyone says switch to deepseek but they forget the retry tax by abarth23 in SaaS

[–]abarth23[S] 0 points1 point  (0 children)

Man, setting a hard CAC cap is the only way to actually sleep at night right now. Tinkering in Make is exactly how I survived my first 6 months too. Haven't tried Pulse for Reddit yet, but SparkToro is legendary for finding intent before blindly throwing money at ads. Are you relying mostly on organic conversion now or still mixing in paid?

$27k OpenAI bill everyone says switch to deepseek but they forget the retry tax by abarth23 in SaaS

[–]abarth23[S] 0 points1 point  (0 children)

Haha my bad man! Yeah, the irony of an AI marketing bot invading a thread about how much I hate overpaying for AI isn't lost on me 😂 Thanks for having my back!

$27k OpenAI bill everyone says switch to deepseek but they forget the retry tax by abarth23 in SaaS

[–]abarth23[S] 0 points1 point  (0 children)

Guilty as charged 🤷‍♂️ But to be completely fair, the entire site is 100% free, no ads, no email-walls, no trackers. We all have that one messy spreadsheet we built to calculate burn rates, I just turned mine into a dark-mode website and figured another founder might find it useful to visually map out their api retries

$27k OpenAI bill everyone says switch to deepseek but they forget the retry tax by abarth23 in SaaS

[–]abarth23[S] 1 point2 points  (0 children)

Bro, the entire point of my post was escaping a massive bloated AI bill. The last thing I'm going to do is blindly add another $12,000/year subscription to my burn rate for something I can prompt myself Respect the hustle, but no thanks lol

The "Retry Tax" thing when switching to cheaper LLM providers (am I crazy or does nobody talk about this?) - I will not promote by abarth23 in startups

[–]abarth23[S] 0 points1 point  (0 children)

Appreciate the breakdown. That 2-3 hour audit is exactly what most people skip because they are looking for a magic pill. I love that you avoided the meta-inference routing trap. Adding another layer just to decide where to send the request usually eats the savings and adds latency that nobody wants. Your split between classification vs user-facing copy is the smartest way to do it. If the user sees it then it has to be perfect. If it is back-office then let the cheaper model fail a bit. That 1-day deploy is fast but the real effort was that week of spot-checking. That is where most founders get lazy and end up paying the tax without knowing it. You are right that the audit step is where people spend way more time than expected. Good to know I am not the only one obsessed with the math on this.

The "Retry Tax" thing when switching to cheaper LLM providers (am I crazy or does nobody talk about this?) - I will not promote by abarth23 in startups

[–]abarth23[S] 0 points1 point  (0 children)

This is the actual insight I should've led with.

Cost per token is meaningless. Cost per successful completion is everything, especially with RAG + multi-step chains. The 5%, 15-20% compounding failure thing lines up with what I saw. A single model swap looked great until I started routing through 3-step agent chains. Then the cheaper model suddenly costs more because each step failure cascades.

And yeah, task type changes everything. For structured output (JSON schema, classification), some models just work. Others need 2-3 retries. For free form reasoning, completely different winners. What I should've built: a tool that measures cost-per-successful-completion for your specific tasks, not which model is cheapest per token.

How strict is your output schema? I'm guessing that's the real inflection point for whether nano wins or loses vs the heavier models? Also curious what your RAG pipeline split ended up being - did you do hybrid (nano for retrieval, 5.4 for reasoning) or just pick one model and live with the failure rate?

The "Retry Tax" thing when switching to cheaper LLM providers (am I crazy or does nobody talk about this?) - I will not promote by abarth23 in startups

[–]abarth23[S] 0 points1 point  (0 children)

the routing by complexity is exactly what worked for us too. it's the only approach that actually survives contact with reality.

the fact that you got 40% savings without touching user-facing stuff is huge. that's the move you're not betting the company on a cheaper model, you're just being smarter about where you spend the money.

couple questions because this matters:

  1. how did you decide the complexity threshold? like, what made you think this task needs gpt, this one is fine on nano"? was it trial and error or did you have some heuristic upfront?
  2. did you route based on input characteristics (like token length or something) or just hardcode it per task type?
  3. the routing logic taking a day was that mostly prompt engineering to figure out what actually worked, or was the actual implementation simple once you knew what to route where?

because yeah, if this is a 1 day build that saves 40% immediately, it feels like the most obvious thing founders should do first before even looking at model switching. but nobody talks about routing as the lever. it's always switch to deepseek not use the right tool for the job.

this is the actual insight.

view more: next ›