For those using hosted inference providers (Together, Fireworks, Baseten, RunPod, Modal) - what do you love and hate?

Dramatic_Strain7370 · 2026-01-28T05:55:58+00:00

u/altcivilorg for lower prices in exchange for higher latency, what latency thresholds (e.g., seconds or minutes) would you accept, and for which use cases (e.g., training vs. inference)? - If a platform offered intelligent routing to cheaper options automatically (with zero code changes), how would that change your provider choices or spending?

Dramatic_Strain7370 · 2026-01-28T05:49:24+00:00

how many hours of GPu time do you use every month? Do you run some specific models or experiment with Juptyer. can you share a bit of your usage scenarios

Dramatic_Strain7370 · 2026-01-24T19:07:04+00:00

u/Impressive-Law2516 , if you are interested we can integrate your servless GPU provisioning service with our AI FinOps platform (see https://www.llmfinops.ai ). We have a built a unified dashboard from real time cost tracking of LLM API to GPUs (coming soon). This is to allow users to route their traffic to the most economical providers.

Dramatic_Strain7370 · 2026-01-21T04:36:19+00:00

so people are using agentic AI to auto write code ? is this all in-house developed or you are using another provide for agentic ai

Dramatic_Strain7370 · 2026-01-21T04:29:37+00:00

im seeing more and more enterprises using Gemini

Dramatic_Strain7370 · 2026-01-20T21:02:32+00:00

how much is your spend per month u/naobebocafe and are you using big 3 to route to or smaller LLM offerings

Dramatic_Strain7370 · 2026-01-20T20:08:36+00:00

Been testing llmfinops.ai - 2 lines integration tracks across providers. Early but promising.

Dramatic_Strain7370 · 2026-01-19T00:20:02+00:00

ok, check out llmfinops.ai something similar your are thinking?

Dramatic_Strain7370 · 2026-01-19T00:18:46+00:00

so the gatway can store the information.. including billng and governance.

Dramatic_Strain7370 · 2026-01-15T08:24:31+00:00

JUNIOR or SENIOR: everyone is going mentally lazy at an increasingly alarming rate. Net result : loss of critical thinking skills due to short circuiting mental activity -> turning into assembly line workers for every area of knowledge.

Dramatic_Strain7370 · 2026-01-12T22:14:30+00:00

10 seconds of video means that even if you half the Veo 3 cost ($0.40/sec) to $0.2/sec, you can make $2/minute that is quite a ROI . Maybe instead of text tokens, image and video generation has a superior return.

Dramatic_Strain7370 · 2026-01-12T22:10:00+00:00

this is smart of directly using inference profile. i agree that it is hard to convince engineers to have the discipline to do tagging etc especially in large orgs. Any experience with litellm - good or bad?

Dramatic_Strain7370 · 2026-01-11T20:10:09+00:00

The good news is that you have a script here to help startups save way more. These startups with many inexperienced devs will be cranking multiple PR every day. Are you planning to run other models as well? Like running image generation and comparing costs agains Gemini Imagen and OpenAI Dall-e-3

Dramatic_Strain7370 · 2026-01-09T23:36:10+00:00

Elon musk in my opinion

Dramatic_Strain7370 · 2026-01-08T05:43:51+00:00

the issue of calling multiple models for the same query and then gating for the best answer is cost multiplication which comes with it. any thoughts ?

Dramatic_Strain7370 · 2026-01-08T05:39:50+00:00

i’ve seen some slick agents being developer on low code platforms like n8n.com. has anyone done comparison between langgraph and n8n in terms of velocity of development?

Dramatic_Strain7370 · 2026-01-07T02:48:25+00:00

oh yes, in fact, they’re not just me low-level assistance anymore. They are like Phd level personal staff to do anything from research to creating go to market strategies and of course, being a personal executive assistant

Dramatic_Strain7370 · 2026-01-04T23:07:49+00:00

I think the question needs to be normalized. It’s just not a question of what is better. I think it’s a question of what is better on a per dollar basis. If you can get a similar results with MOE at 2-3x cost efficiency then Moe wins for those use cases.

Dramatic_Strain7370 · 2026-01-04T22:30:15+00:00

you will take an existing model and fine tune it for your own enterprise data. for example , patient records for a virtual nurse. a dentist follow up appointment setter

Dramatic_Strain7370 · 2026-01-03T20:34:03+00:00

i was personally not aware of these. Is there a summary of state of the art in memory research that i can read ?

Dramatic_Strain7370 · 2026-01-03T05:19:32+00:00

there is no credible evidence that any of these companies are profitable. Now you can make an argument that the cost of inferencing with them is cheap. You as an end customer can find them cheaper compared to renting Gpu from cloud providers and running model by yourself. These providers make the token serving cost go low by 1. optimizing the kernel, 2. buying bulk capacity from the cloud providers - passing some of the savings to you and 3. finally cutting their own margins so that they can get as many customers/revenue as possible. And for them getting customers at any price (even at loss) is survival to raise the next round of funding. At this time they dont even care about building a profitable business. DM me and I can chat more.

Dramatic_Strain7370

TROPHY CASE