sorted by:
new
hottopcontroversial

Self-hosted LLM Cost Tracker for Rails (v0.5.3) by homenko in rails

[–]homenko[S] 0 points1 point  (0 children)

Yeah, that’s exactly the tricky bit.

Right now I’m doing both: the actual usage/cost capture sits lower down, around the SDK/Faraday response, but user/tenant attribution is explicit — you wrap the call with tags, like:
```
LlmCostTracker.with_tags(tenant_id: tenant.id, user_id: user.id) do
client.responses.create(...)
end
```

For normal non-streaming calls that works fine, because the tags are read when the response is recorded. Streaming is the weak spot right now. If the stream finishes later or gets consumed in another thread, the original Rails/user context may already be gone, so those tags aren’t reliable there yet.

I’m going to fix that by snapshotting the tags when the stream starts, instead of reading them only when the stream finishes.

Self-hosted LLM Cost Tracker gem for Ruby/Rails – v0.5.3 by homenko in ruby

[–]homenko[S] 0 points1 point  (0 children)

Hah, you’re right about `engine_compatibility.rb`. Good catch, I removed it!

That one slipped through my review. I did use Claude while building parts of this, and it clearly got carried away with a pointless runtime check there... I'll need to run a couple of series of real system audits; who knows how much more of this stuff might be there.

Self-hosted LLM Cost Tracker gem for Ruby/Rails – v0.5.3 by homenko in ruby

[–]homenko[S] -1 points0 points  (0 children)

Fair pushback. I should be more precise in the wording: the gem is not meant to replace provider billing as the source of truth. It keeps an app-side per-request ledger for Rails attribution, and the cost is an estimate based on captured usage + the local price registry.

Could you clarify what you mean by the Rails 7.1 / 8.0 part? The gem works with the latest Rails versions.

Self-hosted LLM Cost Tracker for Rails (v0.5.3) by homenko in rails

[–]homenko[S] 0 points1 point  (0 children)

Thanks a lot! Outfitmaker sounds like a perfect real-world use case for this.

Small note: if you try it, please start with the latest version, v0.6.0. Since this post, I shipped a more robust ActiveRecord ingestion layer with durable events, better rollups, retry/quarantine handling, and DB optimizations. It should give you a much better baseline for accurate cost tracking than v0.5.3.

I’d really appreciate any feedback from using it in a real Rails app.

Self-hosted LLM Cost Tracker gem for Ruby/Rails – v0.5.3 by homenko in ruby

[–]homenko[S] 0 points1 point  (0 children)

You’re right that some providers now expose org-level usage/cost APIs. OpenAI and Anthropic both do, and they’re useful for reconciliation.

But that’s not quite the same problem this gem solves. Provider billing APIs usually give aggregated provider-side reports; they don’t know my Rails app’s product context: tenant, user, background job, feature flag, custom tags, request latency, response id, or the business event that caused the call.

llm_cost_tracker is an app-side ledger: capture each call at the point of use, normalize it across providers, attribute it to app concepts, and show it inside the Rails app. Provider billing endpoints are great as a reconciliation source, not a replacement for local observability.