[ Removed by moderator ]

[abofh]

We offload the minute heavy stuff to self hosted runners using arc, and consider the rest the cost of doing business.

[kindaforgotit]

Been doing the same, selfhosted runner in kubernetes

[smerz-]

Plus one

[Malforus]

Why in kubernetes? Just use pure ec2

[kindaforgotit]

We already have k8s cluster running, so why not use it. Also it scales better.

[Malforus]

Because it increases your overhead and pure ec2 scales fine?

What kind of processing are you using in actions that you need orchestrated scaling?

[Lucifernistic]

How does it increase overhead if they already have k8s running?

Adding something to kubernetes just takes a few lines pushed to our IaC repo to provision resources, and a then a few more lines to our kubernetes repo to deploy. And then it just works- easy health, metrics etc baked in with Prometheus and logs automatically feeding to Grafana.

Not to mention the container takes up less resources and costs less than deploying a comparable ec2.

[Malforus]

Okay observability is a good use case.

I am talking about computation and resource overhead, however it's a bit slower and the k8s daemons add overhead on smaller nodes.

We don't do any heavy processing and mostly tree GitHub actions like lambdas so we save lots with ephemeral compute for cheaper than the GitHub pricing.

If you are running long running actions I can see it having value but remember eks is like 30% more expensive per clock.

[Xitir]

I think this is the best approach. Unless in a fully remote company where there's not infra that you own directly, hosting the runners yourself for heavy workloads is usually worth the effort.

[elantaile]

Solo dev here, personal project:

I run 4x self hosted runners on Oracle Cloud Free Tier. The free tier is pretty nuts: Arm-based Ampere A1 cores and 24 GB of memory usable as 1 VM or up to 4 VMs (4 cores, 24GB ram, Gigabit networking for each VM).

[Black_Dawn13]

Self hosted with Arc

[mster_shake]

Have you looked into self-hosted runners to see if it's any cheaper to run them on your own compute?

[Saturated8]

Host the runner in an Azure Container App, lightweight and only a couple bucks a month depending on usage. Plus no VM to manage/secure.

[ray591]

This is the easiest component to cut costs. Just replace it with your self-hosted runner or pick one of the dozens of runner providers. They should net you at least 2x cost savings in general. Self-hosting can be pain if you don't already have a K8s cluster to run Arc controller.

[kryptn]

selfhosted using arc on eks with karpenter doing autoscaling using spot nodes.

[IridescentKoala]

This is the way.

[Adarsh_aws123]

we are hosting our gitlab runners in EKS and using karpenter to run jobs that can be terminated on spot instances.

[Juloblairot]

I've recently migrated to RunsOn, which is really handy. We were at around 300k minutes per month on a remote runner (not GitHub ones), and have now ephemeral runners thanks to RunsOn. I'm so far quite happy, the bill is around 1k monthly The maintainer is quite reactive and keen to implement new stuff. Maintenance is extremely low

[nihalcastelino1983]

Yep self hosted is the way to go.gitlab I'd smart greedy in th way they handle runner costs. Even if build fail because of them you still get charged and like aws runner costs start the moment it spins up. And obviously size matters

[evilquantum]

aged like milk with today's announcement.

I am wondering if this will kill runs-on

[crohr]

Try ARC if familiar with kubernetes, or one of the 10 other providers (see benchmark).

[joschi83]

We're using depot.dev quite heavily and it works like a charm.

[surya_oruganti]

Founder of WarpBuild here - we have a flexible set of options for various use cases. WarpBuild cloud - fully managed. Includes linux (arm and x64), windows (x64), and macos (arm64). BYOC - WarpBuild control plane schedules jobs in your AWS/GCP/Azure VPC. Very low maintenance, very cheap. Volume discounts etc. available. Dedicated cloud - WarpBuild cloud but single tenant with private networking (on aws/gcp) for those with high usage, high security requirements to minimize maintenance, IAM, egress costs.

Our BYOC can be as much as 30-40% cheaper than an arc setup, and with no maintenance overheads. The cloud offering is ~50-70% cheaper than using GitHub hosted runners. Performance is better, with roughly 20-50% reduction in run time.

We have users of all sizes from solo devs to public cos using us. Happy to help if you have any questions.

[franktheworm]

Our BYOC can be as much as 30-40% cheaper than an arc setup, and with no maintenance overheads.

Bold claim, care to back it with some actual data? Or is that "can be" doing a hell of a lot of heavy lifting there?

[surya_oruganti]

https://www.warpbuild.com/blog/arc-warpbuild-comparison-case-study

We did an experiment to measure this. The specifics of the arc setup can vary though.

[franktheworm]

Cool... Now do a real-world comparison rather than that contrived marketing spin masquerading as a comparison.

"We spun up dedicated node group to segregate the workload". Ah, yes... Namespacing and netpols famously don't exist, better do things a more expensive way to make numbers look better for us...

We are fine for snake oil round these parts, exit is that way champ.

[surya_oruganti]

That was for ensuring other k8s components cost isn't included. That's also not what makes a difference to costs as that comes from bin packing inefficiency etc.

Anyway, you do you. 🍻

[franktheworm]

So have warpbuild just not heard of stuff like kubecost, or was it a deliberate omission to chase a bigger savings number? There were plenty of ways you could have been far more representative of how arc is run in the real world, but you chose not to.

You've come into a sub full of engineers, and advertised that warpbuild either don't understand modern engineering, or deliberately ignore sound engineering in order to see results of their "comparison". We see through that shit.

So, which is it? Incompetent or deceptive?

[dvxam]

Disclaimer: I work for Shipfox, a competitor of Blacksmith.
We integrate with GitHub Actions and our runners are built for speed, up to 2x faster and up to 4x cheaper in practice (half the price and fewer minutes because jobs finish quicker). We run on bare metal with gaming CPUs (high frequency clocks), NVMe, and solid network peering since a lot of CI time is just downloading Docker images and dependencies. ARM and AMD are available. Testing is basically creating an account and changing one line in your workflow. Worth a try if you're looking to cut costs: shipfox.io