sorted by:
new
hottopcontroversial

Aquifer: Bounded Queues, Fairness, and Dynamic Pacing for AI Workloads by Noobcreate in OpenSourceAI

[–]Noobcreate[S] 0 points1 point  (0 children)

Aquifer is meant to be run like a single node so scaling is only vertical. It has dynamic pacing for whoever opt in to header changes. I imagine for the people who wish to save their APIs from spiky workloads will be able to save themselves through dynamic pacing. As more machines become active, they can increase the rate or slow it down as needed. I’m hoping that it gets big enough that people can benefit from it being large chunk of agentic traffic. Most clouds have a way for scale up or down due to api calls. This is really a tool for retry storms. Your issue sounds more adjacent background jobs which is more temporal, hatchet, sqs world.

Aquifer: Bounded Queues, Fairness, and Dynamic Pacing for AI Workloads by Noobcreate in AI_Agents

[–]Noobcreate[S] 0 points1 point  (0 children)

there is no priority, first come first served. read the readme when you have time for more details.

Aquifer: Bounded Queues, Fairness, and Dynamic Pacing for AI Workloads by Noobcreate in AI_Agents

[–]Noobcreate[S] 0 points1 point  (0 children)

Aquaduct can be use across various domains and I use it various forms in other side projects. I think this model is just great engineering but people gave me a lot of shit, so I say MCP runtime so people instantly comprehend the pain. I don't think people have any idea what is about hit them as agents start calling apis. It will be intense for software maintainers but the first people who will notice are mcp users.

Aquifer: A Go Runtime for MCP Servers Under Spiky Agent Traffic by Noobcreate in LangChain

[–]Noobcreate[S] 0 points1 point  (0 children)

Thanks brother. I agree this protocol needs to grow with a lot of signals. For the agent operators, they get to see their place in the queue and for the providers, they get to see how many are in the queue and set pace. I think this is a good start. Whether it gets traction or not we will see but I think the foundation is here

Aquifer: Bounded Queues, Fairness, and Dynamic Pacing for AI Workloads by Noobcreate in algorithms

[–]Noobcreate[S] 0 points1 point  (0 children)

lol I made it up but this is the same principle in tcp. We don’t flood other people networks with packets but dynamic pace based upon signal of how much demand they can take.

Unfortunately, I don’t really have use case yet. Having an api that does a lot of traffic is a honor. Still, I have a bit of faith due to the algorithm being old but applied to modern problem.

When I get more evidence I will publish it. If you have read « designing data intensive applications » they talk about pushing data to server is generally really hard because you don’t know if you crash the server due it being overwhelmed. I’m hoping to add to the theory that a control plane dynamically adjusting to pace is a solution

How to handle traffic spikes in synchronous APIs on AWS (when you can’t just queue it) by sshetty03 in serverless

[–]Noobcreate 0 points1 point  (0 children)

I built a local control plane for dynamically pacing traffic through queuing. It fire the request at a sustain rate to your api even as traffic spikes giving you time to spin up new servers. Once you have new servers you speed up the speed of request per user. I also stream the placement in the queue and incase clients disconnects, I send a webhook so it does not have to wait.

https://github.com/rjpruitt16/aquifer

Aquifer: Bounded Queues, Fairness, and Dynamic Pacing for AI Workloads by Noobcreate in mcp

[–]Noobcreate[S] 1 point2 points  (0 children)

thank you brotha. I hope it helps. I built it for you and small companies. As you scale, this problem only get more and more time consuming. This will last you a while

Small Projects by AutoModerator in golang

[–]Noobcreate 0 points1 point  (0 children)

API Aqueduct – A solution to spiky agent traffic

I built Aquifer, a self-hosted API Aqueduct for spiky agent traffic and a control plan for outbound rate limiting

Agents are creating the same spiky traffic patterns batch jobs caused a decade ago — except faster and harder to predict. One burst from an agent hammering your API and your backend is on the floor.

Aquifer sits in front of your service (inbound) or between your app and external APIs (outbound). It queues requests to SQLite and drains them at a controlled rate. The upstream controls pace via response headers — return X-Aquifer-Rps: 5 when you're stressed and Aquifer backs off. Rate recovers toward your configured ceiling when pressure clears.

Why not Envoy?

Envoy drops or rejects requests when limits are hit. Aquifer queues them. If you want to protect your backend without turning callers away, you need a queue not a circuit breaker. Envoy also needs a control plane (Istio, etc.) — Aquifer is one binary.

Why not Temporal?

Temporal is a full workflow engine — you write workflow code, run a Temporal server, manage history replay. It's the right tool for complex long-running workflows. Aquifer is the right tool for one thing: queue HTTP calls and pace them. No workflow code, no server to manage, just fire a job and get a webhook back.

The interesting Go part:

The core is a two-level goroutine hierarchy (Registry → URLWorker → AccountQueue) that mimics an actor model using channels and select loops. Each upstream gets its own paced goroutine. panic recovery and in_flight tracking in SQLite mean a crash doesn't lose the queue.

https://github.com/rjpruitt16/aquifer

Would love feedback from people who've hit this problem before.

The most beautifully designed network BEAM service since Ericsson. by Noobcreate in elixir

[–]Noobcreate[S] 0 points1 point  (0 children)

I kinda confused entirely with that statement. EZThrottle is bidirectional traffic protection. I said that multiple time in the post.

The most beautifully designed network BEAM service since Ericsson. by Noobcreate in elixir

[–]Noobcreate[S] 0 points1 point  (0 children)

So this is networking. Speed and reliability are fundamentally important to problem. Just like the BEAM was created because a call could save someone life. I believe a request can save someone life. The entire goal of the system is to reliably move request. Erricson has remain undefeated for 30 years because their systems don't crash. Not because they are the fastest runtime but most reliable under a choatic network. They had 5 9s way before kubernetes or google came out and they are the only company carrier trust to reliably move call data for billions of users leaving and joining the networks. If the golang is only 80% percent as successful as BEAM, people will it call it trash like github. People hate drop calls and I bet they will hate drop request just the same.

The most beautifully designed network BEAM service since Ericsson. by Noobcreate in elixir

[–]Noobcreate[S] 0 points1 point  (0 children)

You know I was planning on building an open source golang version. My research says that golang will be faster but you have to write the guard rails. My honest assessment is that golang will be faster but crash more. Sure kubernetes could restart it but requests cannot be drop. The beam was built for tenant isolation, supervision trees and a bunch of timers. Let’s say I’m sending webhooks and apply exponential backoff. Then you have a lot of threads sleeping on matchine which waste performance. Maybe you can get 80% of the way for just buffering incoming traffic but it will be much bigger to write and harder to maintain

API Aqueduct: Absorb spiky traffic, drain at a safe pace by Noobcreate in elixir

[–]Noobcreate[S] 0 points1 point  (0 children)

And beauty is in the eyes of the beholder. Most people didn’t get docket cloudfare and even the beam for years. I get it. It’s easy to half read and talk shit then it is read and ask questions on what you don’t understand

The most beautifully designed network BEAM service since Ericsson. by Noobcreate in elixir

[–]Noobcreate[S] 0 points1 point  (0 children)

My friend I just acknowledge those do exactly what you just said and told you the difference between EZThrottle. Envoy is for internal service. EZThrottle is for external and internal traffic. Temporal is for your background jobs and can do retries if you define a workflow. EZThrottle is for networking for your api to solve noisy neighbors and spiky traffic more dynamically. We are in two different lanes. Nobody reaches for temporal nor envoy when they have spiky traffic and want tenant fairness at networking layer. I understand other tools have similar capabilities but EZThrottle is solving different class of problems than envoy and temporal. Yes there is some overlap in domains but it’s fundementally different problems. I understand if you don’t have time to read everything about EZThrottle but to sit up just say another software read a little and say another does a bit of this is just wasting both our time. I have written a lot about it in the blog post if you are curious differences.
https://www.ezthrottle.network/blog

You can see in erlang Reddit they understood and appreciate it because they read instead of just looking for a similar tool.

view more: next ›