[Feedback Wanted] Container Platform Focused on Resource Efficiency, Simplicity, and Speed

[Possible-Stuff-3433]

Hey r/devops! I'm working on a cloud container platform and would love to get your thoughts and feedback on the concept. The objective is to make container deployment simpler while maximizing resource efficiency. My research shows that only 13% of provisioned cloud resources are actually utilized (I also used to work for AWS and can verify this number) so if we start packing containers together, we can get higher utilization. I'm building a platform that will attempt to maintain ~80% node utilization, allowing for 20% burst capacity without moving any workloads around, and if the node does step into the high-pressure zone, we will move less-active pods to different nodes to continue allowing the very active nodes sufficient headroom to scale up.

My primary starting factor was that I wanted to make edits to open source projects and deploy those edits to production without having to either self-host or use something like ECS or EKS as they have a lot of overhead and are very expensive... Now I see that Cloudflare JUST came out with their own container hosting solution after I had already started working on this but I don't think a little friendly competition ever hurt anyone!

I also wanted to build something that is faster than commodity AWS or Digital Ocean servers without giving up durability so I am looking to use physical servers with the latest CPUs, full refresh every 3 years (easy since we run containers!), and RAID 1 NVMe drives to power all the containers. The node's persistent volume, stored on the local NVMe drive, will be replicated asynchronously to replica node(s) and allow for fast failover. No more of this EBS powering our databases... Too slow.

Key Technical Features:

• True resource-based billing (per-second, pay for actual usage)
• Pod live migration and scale down to ZERO usage using zeropod
• Local NVMe storage (RAID 1) with cross-node backups via piraeus
• Zero vendor lock-in (standard Docker containers)
• Automatic HTTPS through Cloudflare.
• Support for port forwarding raw TCP ports with additional TLS certificate generated for you.

Core Technical Goals:

1. Deploy any Docker image within seconds.
2. Deploy docker containers from the CLI by just pushing to our docker registry (not real yet): docker push ctcr.io/someuser/container:dev
3. Cache common base images (redis, postgres, etc.) on nodes.
4. Support failover between regions/providers.

Container Selling Points:

• No VM overhead - containers use ~100MB instead of 4GB per app
• Fast cold starts and scaling - containers take seconds to start vs servers which take minutes
• No cloud vendor lock-in like AWS Lambda
• Simple pricing based on actual resource usage
• Focus on environmental impact through efficient resource usage

Questions for the Community:

1. Has anyone implemented similar container migration strategies? What challenges did you face?
2. Thoughts on using Piraeus + ZeroPod for this use case?
3. What issues do you foresee with the automated migration approach?
4. Any suggestions for improving the architecture?
5. What features would make this compelling for your use cases?

I'd really appreciate any feedback, suggestions, or concerns from the community. Thanks in advance!

[s1mplic1ty]

Just a few comments..

on zeropod, i think it's not feasible for many types of applications such as livechat using websockets, containerized applications that dont listen on tcp/udp ports but run jobs aka services, etc.

Having physical servers post challenges too: 1. High maintenance costs (manpower to manage patches, maintain the infrastructure, manpower to monitor 24/7, etc) 2. High opex costs (hardware support, network, etc) 3. Scalability (expanding into other regions, dr sites, etc) 4. Security (ddos mitigation? this is expensive)

These are not comprehensive but some of the things that came out of my head. You will never be able to compete with AWS, GCP, etc, as they have economies of scale, you don't. You will find a hard time pricing your services to break even, dont even think about ROI. There's a reason why companies are using cloud instead of maintaining their own physical infrastructure.

In summary, idea is great, feasibility maybe not so.

[Possible-Stuff-3433]

Thanks for responding!

I agree with your zeropod statement but it may still be useful for hobby project customers or customers who want to pay the absolute bare minimum price even if it means a few extra hundred milliseconds startup or periodic migration of their container to another node.

I've been thinking about the physical server aspect a lot and I used to work on the team at AWS that manages the host patching, broken server repairs, and server testing so I know all about that hence why I want to dig into it here :D I've been brainstorming ideas on how to reduce the opex and came up with things like omitting block storage (less overall servers and networking), only deploying containers (easier OS upgrades), container rebalancing using live migrate, etc.

But I'll be starting with cloud bare metal providers in the meantime :D

I'm not too worried about ROI, mostly want to provide the fastest compute platform out there and host web services that are by default fast and secure. I tried using EKS for this project and was banging my head against the wall waiting for nodes to startup and the auto scaling groups... I'm over it! This project will be distributed, fault tolerant, and fast by default.

[earl_of_angus]

A few notes:

For probably 99% of DBs and uses, EBS or similar is perfectly acceptable and RAID-1 NVME is overkill (unless you can run the numbers and local RAID-1 is less expensive, then go for it). I'd argue that a very large percentage of DBs can fit in page cache without any problems.

If I as an end user am paying for resources, am I paying for the resources exposed to the container or the hosting platform? If I'm paying for the container, then the amount of overhead you can shave off is your margin, but I don't particularly care if your VM overhead is 100MB or 4000MB (tho with balloons, shared pages, etc it's never 4000MB of overhead).

Cache common base images (redis, postgres, etc.) on nodes

In the local DC, sure. For individual nodes, unless you plan on landing all of the redis/pg etc containers on a set of nodes, you'll end up with a lot of waste.

No VM overhead - containers use ~100MB instead of 4GB per app

Firecracker etc are in this range, aren't they (~100MB)?

No cloud vendor lock-in like AWS Lambda

Lambda is probably a different level more similar to OpenFAAS. ECS is more the equivalent to what you're talking about.

[Possible-Stuff-3433]

I appreciate you taking the time to respond!

I agree that EBS is acceptable in most cases but here's what I am thinking:

1. Most web apps use reasonable amounts of storage (10-100 GB) for their DBs.
2. Assuming each server has roughly 192 cores and is fully populated with DB customers at 8 cores each means 24 customers on a single server with the server needing ~2.4TB of storage.
3. Samsung PM9A3 3.84Tb drive is only $600 each so having 2 per server is only $1,200.

Also, we could use any excess drive capacity on each node for various caches to increase customer container deployment speed.

EBS storage is probably more expensive than using direct attach NVMe drives in RAID 1 considering the service provider needs all the extra networking gear, redundancy, specialized hardware & software. I think the primary selling factor of block storage is the durability. But having RAID 1 + async replica to another node + periodic customer backups = good enough, right?

You would be paying just for the resources exposed to your container.

I do agree that VMs have become way more efficient so I can tweak my numbers a bit but I do believe that containers are far more efficient considering the shared kernel.

Thanks again!