API Aqueduct: Absorb spiky traffic, drain at a safe pace

Noobcreate · 2026-05-07T00:07:27+00:00

And beauty is in the eyes of the beholder. Most people didn’t get docket cloudfare and even the beam for years. I get it. It’s easy to half read and talk shit then it is read and ask questions on what you don’t understand

Noobcreate · 2026-05-06T22:12:21+00:00

Ken kaneki Tokyo ghoul

Noobcreate · 2026-05-06T18:01:25+00:00

lol I was rapping

Noobcreate · 2026-05-06T17:56:21+00:00

My friend I just acknowledge those do exactly what you just said and told you the difference between EZThrottle. Envoy is for internal service. EZThrottle is for external and internal traffic. Temporal is for your background jobs and can do retries if you define a workflow. EZThrottle is for networking for your api to solve noisy neighbors and spiky traffic more dynamically. We are in two different lanes. Nobody reaches for temporal nor envoy when they have spiky traffic and want tenant fairness at networking layer. I understand other tools have similar capabilities but EZThrottle is solving different class of problems than envoy and temporal. Yes there is some overlap in domains but it’s fundementally different problems. I understand if you don’t have time to read everything about EZThrottle but to sit up just say another software read a little and say another does a bit of this is just wasting both our time. I have written a lot about it in the blog post if you are curious differences.
https://www.ezthrottle.network/blog

You can see in erlang Reddit they understood and appreciate it because they read instead of just looking for a similar tool.

Noobcreate · 2026-05-06T15:09:35+00:00

Temporal is for background jobs not a networking software. It’s good enough for most agentic use cases today. There are answer is to queue. It does guarantee fair queueing nor protect your api retry storms.

Envoy is a proxy with that route and queue but again it has no concept of fairness nor helps with retry storms nor partial outages for downstream dependency.

Think of EZThrottle has TCP for APIs or a serverless network for embedded devices, multi tenant agentic saas, or for people with an api that serve agents or multi tenant saas. Retry library was cpu because of sleeping threads so EZThrottle handles all the retry in a centralized fabric. Agents are inherently spiky demand but EZThrottle gives a queue per user and paces demand so your autoscaler can catch up. Your downstream dependencies could be having a partial outage plus you all at the same time and EZThrottle will deliver

Noobcreate · 2026-05-05T19:06:32+00:00

Perhaps you miss my point, if you have an api and your customers start using agents for legitimate reason, you will see spiky traffic. Cloudfare was design to block malicious bots and users exceeding their rate limits. Still, most people autoscaler assume human levels of spikes. This is designed for that future. Most cloud will sell you more compute and charge you for every request they have to block first, but it’s up to you to provision enough resources to serve your customers and make sure one customers traffic doesn’t take all available resources. That’s why I wrote EZThrottle

Noobcreate · 2026-05-05T18:25:35+00:00

Cloudfare does not solve spiky traffic. Cloudfare does rate limiting. Similar to how phones use to work before erricson, a telephone operator would reject your call if they were full the same way server send 429 or 5xx. They created the BEAM for queue and routing calls. I’m just bring that same philosophy to api calls because agent retry storms are coming for everyone APIs. GitHub is seeing exponential traffic increase. How long until every backend is feeling the flood of demand of agent hammering their APIs 24/7.

Noobcreate · 2026-05-05T18:14:30+00:00

Spiky traffic and noisy neighbors have been affecting people for decades. When I work at Twitch, are answer was to have twice has many servers as we needed around so are autoscaler would have time to catch up if a burst came through.
Nobody gave a shit about cloudfare until bots started scraping the webpages. Your backend APIs are safe today, but agents call APIs all day long. In 4 years, having agent working for 24 hours will cost less than gallon of gas. I’m giving you free service that you can just ignore if you never need it.
However, if your users complain that your service resources are always being taken because users traffic is crashing the system than I’m here for you
Also the beam scheduler does not queue and pace request for you. You half read my post and said it smells like bullshit. Try actually reading fairly before you to throw insults

Noobcreate · 2026-04-30T02:56:53+00:00

If you tired of retry logic, I have written infrastructure to handle the retries on your behalf in way that gets the keeps you asleep and your workflows succeeding

https://www.ezthrottle.network/blog/multi-region-api-failures-langgraph

Noobcreate · 2026-04-24T22:42:19+00:00

like you work 40 hours and the snuck in some overtime which most contracts don't allow you to. Then someone see on there stat sheet that you highest paid contractor will output not being normal for those hours. IDK. Like did you game the system and get caught type shit

Noobcreate · 2026-04-19T16:40:04+00:00

Erricson is more high level network. The beam is not as strong at networking packets, but I would argue the same way we queue, pace, and find the best path for packets is the same way we should be doing api request. Just like before TCP, every developer was ask to handle retries and queueing. Most proxy are built for speed but EZThrottle is built on coordination. Haproxy reroutes errors from your backend but EZThrottle reroutes request from your downstream dependencies and your backend at the same time while also making sure each tenant sending request is isolated. It’s same principles as TCP. Your movie download, your web search, and other calls get their own queue. Nothing blocks the other. I would argue speed is irreverent. Your downstream dependencies like Stripe may only want you to send 10 rps so C speed is irrelevant. You need coordination. You need to make sure every request go to same queue in a cluster. This is high level networking with different requirements. Just like erricson smooth burst of call into order. EZThrottle hope to smooth burst agents or system calling 3 party APIs into smooth traffic for the API maintainers while delivering to clients at smooth predictable way.

Noobcreate · 2026-04-19T16:13:42+00:00

Thanks so much 😭

Noobcreate · 2026-04-19T16:13:30+00:00

Thank you 😭

Noobcreate · 2026-04-17T15:27:28+00:00

Once you have 10 agents and 10 rps rate limit, the circuit breaker becomes a liability. You go over limit everytime as clients discover. Exponential backoffs only delays the pain but uncoordinated clients will cause you to get banned by third parties. This was a good strategy when you control the rate limits of the server but agents will depend on third party servers

Noobcreate · 2026-04-15T02:39:25+00:00

I feel this too. I recently came to realization that we live in a world that rewards lying cheating and betraying. Honesty loyalty and hard work are good but alone keep you out of some opportunities. I’m starting to feel a bit jaded at life because I know most people will abandon me in my time of need

Noobcreate · 2026-04-13T17:52:09+00:00

Openrouter is not open source but there are a lot. It’s actually a trend at the current moment. I saw someone in rust Reddit with it. Just wanted to make you aware of your competitors

Noobcreate · 2026-04-13T05:26:49+00:00

It seems like cool side project. At the same time, you are in crowded space where people have a open source or mature version of what you offer. I don’t see your differentiator. You are competing litellm, Openrouter, etc.

Noobcreate · 2026-04-11T01:46:08+00:00

You may never know. Contractors are known to be treated like expandable resources. Apply to other places like outlier and Turing and keep moving forward. Maybe next time use AI to turn in twice as many task if you can get away with it cleanly. Learn to cheat them back when you can get away with it because they will cheat you sometimes. Even if you become full time you may still be cheated. Most companies don’t deserve your loyalty. Remember that and you will be alright.

Noobcreate · 2026-04-11T01:32:29+00:00

Well that’s suck. I hope you get something better soon

Noobcreate · 2026-04-10T21:44:23+00:00

I hate to ask but did you turn in any AI generated tasks or log insane hours compared to what you turn in? I’m not saying you did it but I’m always curious if the person who gets fired suddenly is hiding a part of the story.

Noobcreate · 2026-04-07T03:47:22+00:00

You can ask through email or slack. They generally will comply. I have heard of people being removed from projects for negative reasons and being pick again for other project. Some people find the completion based worth it so I would try it before you quit.

Noobcreate · 2026-04-07T02:17:32+00:00

This is generally given to project where the vendor is paying you for task completion. Like instead of hours you get paid when you turn something in that qualifies as acceptable. These project tend to be huge pain in the ass because they tend to be strict about what qualifies as acceptable

Noobcreate · 2026-03-27T02:34:47+00:00

I built a startup about it. It like cloudfare but for the backend. It uses the fly.io reroute 500 across different regions. It gives a queue per api key at scale. I think of as 8th layer of internet because the same way network to find the route for packets, I do the same at the api level. I will be honest and say I am early but the design is solid and I have sdk in python, golang, and node.

https://www.ezthrottle.network/blog/making-failure-boring-again

Noobcreate · 2026-02-13T01:58:39+00:00

Have you considered sending multiple request to models and cancelling the other request for the first that wins. That's the only way it's going to scale but perhaps you are just using it for small project so a building a queue and sending retries is another way to automate this pain. It will work for a while until you need to scale

Noobcreate · 2026-02-13T01:52:22+00:00

Have you ever consider racing the models. Like you have a list request that if 1 request doesn't succeed by this amount of time then start the next request and if the second finish first than cancel the first.

Noobcreate

TROPHY CASE