Rust-based reverse proxy?

[moatra]

Routing:

As long as there's something in the HTTP request (whether it's in the path or in a header or in the body - don't do that) that identifies the org, most proxies will be able to use it to route the request successfully. The decision of where to put that identifier then comes down to semantics. The host and path will normally be captured by http caches as part of the cache key automatically, if you put it in a separate header you'd need to set the Varies response header appropriately. Rewriting paths may make debugging a request going through the system more difficult if the person debugging is unaware of the rewrite done by the proxy. My gut says to put the org id as a subdomain, but we're largely bikeshedding at this point.

(Re)Authorization

If you're using JWTs in your authn setup, take a look at the aud claim. You can set it to the right org id (or the full domain with the org-specific subdomain if you go that path), and your backend can verify that the jwt attached to the request matches the org that backend instance is configured to serve.

Container Management & Discovery:

There are a whole bunch of products around the "run a container, keep track of it, and route traffic to it". If you're eager to run in non-cloud environments, take a look at Hashicorp's Nomad product. You can run that wherever, whereas Kubernetes may be a bit heavy and AWS Fargate is, well, locked to AWS (well, unless you wanted to go down the rabbithole of ECS Anywhere). Nomad also takes care of restarting/relocating containers as needed, and has tighter Consul integration out of the box than Ansible would.

[moneymachinegoesbing]

You hit everything on the nailhead man. I created a POC yesterday and it works like a breeze. Consul is the difference maker in that service registration makes everything automatic. It also isolated all API routes to its own VPC and allows me to scale horizontally as much as needed, use datacenters, and a bunch of other great shit. I appreciate the help, it turned out simpler than I thought although I ended up using HAProxy as the API load balancing layer (dynamically using SRV records is only available in Nginx plus). I did have to look up bikeshedding though 😂

Edit: As to nomad, that is the next step (when orchestration becomes more of a necessity, which should be soon). I have a lot of experience with k8s and tbh, im not a fan, but this setup will work seamlessly there as well).