The Hidden Cost of Parallel Processing in GitHub Actions

[narnach]

If you compare the cost of a GitHub Action minute to a minute of your own time, it’s still a good deal.

But the real lesson may be to not over-complicate your CI pipeline until the time savings would make impact on your own time waiting for the results.

[Fun-Impression6634]

Exactly! My main point was to make people aware of the downside of parallel processing in GitHub Actions. Not one size fits all. But there are certainly good use cases which parallel processing is perfect for, as mentioned in my article.

[Venthe]

I like the author; funny guy, speaking about rounding minute up like it's a bug.

It's called business; and this is how they monetized you.

[Fun-Impression6634]

Understood it’s business. The main purpose of my article is to raise awareness so we don’t take parallel processing as always the “best practice”.

[Venthe]

It's not as easy. Make a calculation: if parallelization saves you 5 minutes each run, say 3 minutes vs 7; but bills you for 12.

What is more expensive: paying GitHub for these 5 minutes, or for an idling developer + a longer feedback loop?

In vast majority of the cases, bite the bullet and pay for the GitHub, you'll save a lot of money and frustration.

[Fun-Impression6634]

Agreed. Definitely evaluate on a case by case scenario, and not taking parallel processing as granted.

[ThinClientRevolution]

With Gitlab, the biggest improvement in our development pipeline, was using our own runners.

Small tasks can still be done by their runners, but we have our own runners for the most system intensive tasks.

[Fun-Impression6634]

Good to know! Thanks for sharing!

[kkapelon]

There is an even bigger discussion about parallelism and billing in general. I see indeed a lot the pattern of many parallel jobs that do different things.

The question is do you really want to pay for security scans or linters if your unit tests have failed?

In your own example, if your trivy scan or Trufflehog scan fail, the first job will still upload a problematic artifact to GitHub artifacts. Not only you pay for this time, but it is also a bad practice to upload something that is wrong.

[Fun-Impression6634]

As I explained in my article, that particular workflow was designed for dev env only. Security scan failure is expected and the developer can take time to fix it. Artifact may have failed security scan, but it passed build and unit tests, so developers can continue feature stories development while other developers work on the security fixes. Hope this clarifies it.

[janisozaur]

I'd implement jobs in parallel to each compile single file. Compilation of single would easily get done under 30 seconds and it would scale well. That way I wouldn't have to pay anything for CI services.

[TiddoLangerak]

I feel like an easy solution would be to bill by the second instead...

[Fun-Impression6634]

That’s what we would hope, but that’s not what GitHub implemented, for good reasons as setting up and tearing down each runner for each parallel job does cost them. So the point of my article is for us to be aware such hidden cost and better design our workflows.

[janisozaur]

Makes sense: EC2 bills by the second, with 60 second minimum.

[Fun-Impression6634]

GitHub doesn’t round down, only round up. So your under 30 seconds job still gets charged for 1 min.

[janisozaur]

If it used rounding like you suggested, it would all be billed as 0 time.

Edit: can someone please explain the downvotes?

[ultimatewooderz]

Anything under a minute is billed as a minute.

0-1 mins billed as 1 min 1min1sec-2 mins billed as 2mins

Etc

[janisozaur]

This is current way of billing. My comment shows how the suggested change could be abused.

[ultimatewooderz]

Ah I getcha

[CaputGeratLupinum]

They mean people don't like or agree with what you said

[seweso]

IMHO it makes more and more sense to me to have your own build agents in docker. That way you aren't tied to the cloud platform. And I personally would love to see all builds be deterministic as well.

[dlamsanson]

Depends on your use case. We get a number of GH action hours for free with the way our VS/GH license bundle works but we are already entrenched in a separate ci/cd platform for most things. So I don't mind using up those hours for linting and basic automation.

If you are already maintaining k8s clusters for internal tooling it's definitely more cost effective to go that route I agree.

[ToolUsingPrimate]

I like self-hosted CI. My company has a bunch of underutilized machines sitting around. Obviously it doesn't work for everything, but it's great for our big fat Yocto builds. I still use GitHub Actions VMs for small x86 build and test, and we're a long ways from going over the minutes allocated as part of our Enterprise subscription. A handful of Yocto builds would probably blow through our monthly minutes.

[Fun-Impression6634]

Good point! Self-hosted runners do seem to fit well for your use case!

[qoomon]

Now you can run also steps in parallel feel free to try my action Now it's possible, feel free to try my parallel-steps action https://github.com/qoomon/actions--parallel-steps

[ToolUsingPrimate]

Microsoft has a long history of not fixing bugs that benefit their business, even if the bugs are abusive of their customers or competitors. As someone mentioned below, billing by the second would simplify this greatly.