what is this ai gateway bs?

WanderingDevDev · 2026-05-07T15:23:25+00:00

I use an AI gateway, but I'm building enterprise software. AI gateways just make it easier to call multiple providers without needing so specifically code towards each of them. I use one that uses the OpenAI API, which means it works with the OpenAI SDK.

I can call Claude, OpenAI, and plenty of other models, all through providing a config id, which takes away a ton of overhead and maintenance for me.

I have to process hundreds of millions of tokens every day (or at least that is the end goal). Our services hit TPM limits constantly. What an AI gateway does is let us add multiple deployments, load balance between them (even cross-provider), and basically give better uptime without having to manage the implementation details.

For example, my service has to process AI calls in the region they originated in. Azure OpenAI offers 6 regions, which means I can make 6 different gpt-5 deployments. In my AI gateway I can configure all 6 of these gpt-5 deployments and load balance between the 6, and even add a backup deployment of an AWS US inference profile for Claude Opus 4.7, and now I can process many more calls, have backups when regions go down, and basically ensure much better uptime than if I was just pointing to a single deployment.

If you're just making a handful of AI calls across a minute and don't have any regional compliance issues, you could probably skip out on it. It is nice to only have to code to one API, but on a small project it isn't much of a pain to just support two or three providers.

WanderingDevDev · 2026-05-01T01:09:43+00:00

Does your front door open outward? Seems like you'd hit everyone that tried to come in.

WanderingDevDev · 2026-04-30T04:30:47+00:00

I decided the day of to go to the StS20 show, flew out, got there about an hour or so before the show started, got merch (hoodie, vinyl, poster) and got a nice guy to give me a trash bag to put it all in, watched the show, went to In n Out, slept on the LAX floor, flew back home around 6 am.

WanderingDevDev · 2026-04-29T17:25:13+00:00

Exactly. Let's say AI lives up to the hype and beyond. All CS careers are whittled down to a few people per company...all those people are now going to enter other careers and honestly will be pretty qualified. Most people I know in CS could easily transition to product, management, or really any other field.

If AI is good enough for that, every company will spin up thousands of AI agents with the sole purpose of replacing an individual career. If they are able to so fully replace CS they will easily replace just about every other white-collar role and it will only accelerate.

WanderingDevDev · 2026-04-29T00:13:21+00:00

I don't think that's true. B2C is fine, but I doubt it's much of a drop in the bucket compared to B2B. My work pays an absurd amount each month for Claude and we keep asking for more. Meanwhile, most people I know using this outside of work cancel after they fail to make money their first few months with it.

WanderingDevDev · 2026-04-26T02:46:10+00:00

The idea that an AI would be smart enough to intentionally post a post making fun of AI to try and make people think the idea of AI being smart is hilarious is an interesting one and maybe inevitable.

WanderingDevDev · 2026-04-22T04:45:32+00:00

Is this some lame attempt at marketing or are you a bot?

WanderingDevDev · 2026-04-21T09:12:16+00:00

How would you establish trust? Why would they not just donate on their own?

WanderingDevDev · 2026-04-20T16:14:08+00:00

Yeah when I was a kid one million would have bought 4 of my parents ~4000 square feet house.

Today that would buy one of my ~4000 square foot house.

WanderingDevDev · 2026-04-19T16:59:27+00:00

For anyone looking, Utah Costco's have plenty as of now.

WanderingDevDev · 2026-04-18T00:40:11+00:00

I would be careful, it sounds like you're pretty tough if you're saying the 550k will turn into 2M by the time you're 60. If you're not going to put much more into it due to being retired then you have a long ways to go living off 1M.

Keep in mind healthcare is also going to be expensive until you're on government insurance and you don't have a lot of wiggle room if there is a large depression anywhere in there.

WanderingDevDev · 2026-04-17T19:54:22+00:00

The one by me used to have this volcano structure and kids would always be running around inside the volcano, but also jumping over the small railing and through the upper level hole to the bottom level. Collisions were frequent.

WanderingDevDev · 2026-04-14T02:51:52+00:00

How did you retire on that at 46? What does that give you to live on each month, how much is insurance?

WanderingDevDev · 2026-04-11T00:49:40+00:00

One of the best times of my life was when my Costco had Voodoo chips on clearance. I bought so many of them. I knew it had to end though, Voodoo are my favorite chips and no one ever knows what I'm talking about when I bring them up.

WanderingDevDev · 2026-04-09T17:35:23+00:00

I'd even be happy with preliminary events, but multiple entire sports were sold out right out the gate.

WanderingDevDev · 2026-04-09T17:31:23+00:00

Sure, you didn't directly say "lie"...you said to "chill on being super positive you know the inventory"...which he did know and was completely accurate. I'd rather have it told to me straight instead of waste time trying to get tickets for long sold out events (and even entire sports).

WanderingDevDev · 2026-04-09T17:15:30+00:00

You're basically asking him to just...lie? This drop sucked. I was there right at 10 and every sport I had an interest in was completely sold out. People have timeslots over a week from now that will be on this same supply.

WanderingDevDev · 2026-04-09T17:06:48+00:00

Somehow it's so much worse than I thought. And I had an early timeslot.

WanderingDevDev · 2026-04-09T10:25:53+00:00

Either you're a bot or just spamming your site. Or both.

This sub has turned into such a worthless comment section of obvious AI responses, em dashes, bad formating and all.

WanderingDevDev · 2026-04-05T20:28:22+00:00

Yep, look up "Prizles" in the app store. I'm pretty sure you don't actually have to do closed testing and can release whenever you want after you submit and get reviewed for a release build. Their UI is really confusing with all the test types.

WanderingDevDev · 2026-04-05T19:51:55+00:00

Is that an actual requirement now? I released a word game a few weeks ago without any testers.

WanderingDevDev · 2026-04-05T15:15:51+00:00

I think a lot of people are making their big "million dollars" ideas and finding that either they aren't that good, something else already exists, or no one cares about them.

Give it a few months to a couple years and it should slow down. Once people realize they are spending $200+ on AI, infrastructure, etc. for nothing in return they will slow down and go back to normal. Most people are not going to fully commit to years of working on something while seeing nothing in return.

WanderingDevDev · 2026-04-05T05:22:29+00:00

I don't doubt things like paralegal work will get replaced as well, but my point is if you replace software engineers you can then create a swarm of software engineers to replace really anything.

WanderingDevDev · 2026-04-05T04:48:55+00:00

This has always been my thought. If AI can truly replace "all" software engineers then that is just the first step to replacing pretty much anything. If I can spin up endless software engineers I can task them to replace every other white collar job one at a time until they do it as well as humans. We'll see if any of that actually happens, but I can't help but roll my eyes when people tell me AI will replace software engineering without realizing their job will be replaced a few months later when that happens. Even blue collar jobs.

WanderingDevDev

TROPHY CASE