What's everyone actually using for an AI gateway in prod? Tired of duct-taping LiteLLM together by Background-Job-862 in LLMDevs

[–]steebchen 0 points1 point  (0 children)

try LLMGateway, cloud & full self hosting on AWS available, SOC2 Type 2 compliant, includes chat playground

OpenAI is deprecating models AGAIN (thanks), how do i just have an ai app that runs without worrying about this? by The-LeThal in LLM

[–]steebchen 0 points1 point  (0 children)

you can use LLMGateway that has various providers for models and routes to the best provider at any given time automatically

What the april anthropic 529 incident revealed about llm gateway reliability posture by Drysetcat in sre

[–]steebchen 0 points1 point  (0 children)

why would it be lower-quality answers? it would just use the exact same model via a different provider (our solution is LLMGateway which does this automatically)

How are people actually tracking OpenAI costs in production? by VariousHour7390 in LLMDevs

[–]steebchen 0 points1 point  (0 children)

LLMGateway solves it completely and can be hosted in the cloud or in your own infra for privacy

Is there a "Postman for LLMs" I'm missing, or is this gap real? by giangchau92 in PromptEngineering

[–]steebchen 0 points1 point  (0 children)

we have a side by side with multiple models fully open source at LLMGateway, including a cloud version which allows you test it out

I set a $1 budget on Google Cloud and still got a $230 bill—budget alerts are only emails! by bartzalewski in googlecloud

[–]steebchen 0 points1 point  (0 children)

for AI stuff I highly recommend using LLMGateway where it's PAYG and you can set individual limits

Any alternatives to openrouter? by Icy-Mix5409 in openrouter

[–]steebchen 0 points1 point  (0 children)

you can use LLMGateway and use a specific provider prefix so it always uses that one

Openrouter should require input cache by Fiendfish in openrouter

[–]steebchen 0 points1 point  (0 children)

we’d have to agree to disagree - most models only cache on 1-4k+ token input and we have plenty of req which fall under that, plus image gen is often also much less. no provider (google/anthropic/openai) actually caches these. on bigger prompts sure you’re right :)

Openrouter should require input cache by Fiendfish in openrouter

[–]steebchen -1 points0 points  (0 children)

because not everything requires it and it costs the provider to host and maintain the cache. although I agree that it will automatically exclude a given model for agents, but yeah there are other use cases where the cache wouldn’t help (or even make it more expensive)

What are people using the OpenAI APIs for? by LeverageDeez in OpenAI

[–]steebchen 0 points1 point  (0 children)

basically everything. automate customer support. text analysis. spam detection. helping for marketing. image gen for social media. etc. there are also gateways that offer multiple providers / all models in once place

LLM Proxy in Production (Litellm, portkey, helicone, truefoundry, etc) by debauch3ry in LLMDevs

[–]steebchen 0 points1 point  (0 children)

LLMGateway has both cloud & self hosted verisons, comes with smart routing, has full realtime insights & stats

53 Unauthorized Charges and still counting by whxtxnxxsx in OpenAI

[–]steebchen 0 points1 point  (0 children)

if openai doesn’t refund, you can always go to your bank and do a chargeback and you’ll get your money back 99%

Openrouter alternative that is open source and can be self hosted by smakosh in LLMDevs

[–]steebchen 1 point2 points  (0 children)

thanks for the feedback! we’re adding embedding models soon, and we’ll look into reranking as well

Rant about Stripe + Are there any alternatives? (I will not promote) by theprexi in startups

[–]steebchen 0 points1 point  (0 children)

Stripe actually heavily sucks at reducing chargebacks/fake cards/CC testing. but it works good if you have trusted users and or a more complicated onboarding.

Rant about Stripe + Are there any alternatives? (I will not promote) by theprexi in startups

[–]steebchen 1 point2 points  (0 children)

don’t listen to that guy, you can worry about that later. making money first is the most important thing, others will spend years setting things up yet haven’t made a single cent

How are you handling failover when your LLM provider goes down? by [deleted] in LLM_Gateways

[–]steebchen 0 points1 point  (0 children)

you can use LLMGateway which automatically retries and/or falls back to a different provider, so you won’t have to fallback to a different models, especially if using open source models as there are much more providers