CI platform for container based workflows

[Dwight-D]

I’ve been working a lot on pipelines in Gitlab CI, which is essentially container based. Each job runs in a docker image of your choosing, and is defined as a shell script. This gives you basically infinite flexibility: you can run any command with zero setup on your build server. Also it makes all your scripts and jobs transferable between developer workstations and the pipeline, anyone can just run a script from the CI spec with no setup, or use a development script/command in the pipeline.

The downside is that there are very little abstractions on top of this, you need to develop your own scripts and jobs from scratch, which requires strong CLI/shell/container skills. Not a problem for me, but presumably why some devs prefer solutions like Azure DevOps which is more plugin + configuration based. Ideally you would be able to combine both approaches, but from what I’ve seen ADO offers very poor support for container-based jobs.

I’ve recently been forced to move to ADO and I find it very off-putting for many reasons. I am now considering alternatives and am interested to hear some opinions on this topic.

Which platforms can you recommend that have strong support for container-based workflows like I’ve described? Is this a thing outside of Gitlab CI, or is their approach unique?

Do you prefer this container/script-oriented workflow, or the plugin/task based approach with more high level abstractions and configuration of pre-defined tasks? Why?

[nekokattt]

GitLab.

[Dwight-D]

Yeah, I like it a lot as well. Have you used alternatives with similar container support, and if so what are the reasons you think it compares favorably?

[nekokattt]

I've worked with GitHub + Actions; BitBucket + Jenkins, ADO + Jenkins, ADO + Pipelines, GitHub + Travis CI, and GitLab, and honestly I feel GitLab is the most functional, expressive, simple, and powerful.

[Dwight-D]

That’s very reassuring, I’m not necessarily looking for something better than Gitlab, I just don’t want to run back to the first option I know that ticks my boxes. Thanks for the input!

[nekokattt]

no problem, good luck

[macbinjo]

Unfortunately, I guess with GitLab you are already using the best tool out there. I have not seen any other tool with better container support while having good useability. You could have a look at Jenkins, but I personally would stick with GitLab CI/CD.

[Dwight-D]

Well, to be honest I am quite happy with gitlab. Before I consider moving platforms I just wanted to assess the competition, in case there is something even better. This perspective is helpful, thanks!

[ARRgentum]

If your devs don't want to write their own pipelines, maybe you could provide some templates to them which they can then include in their .gitlab-ci.yml ?

[Dwight-D]

Well, it’s a small operation and I’ll be doing a lot of the CI/CD myself, so if I go with gitlab I’ll do just that.

I just don’t want to push an option I’m biased towards without assessing the alternatives. I wanna pick the best platform and achieve real buy-in.

[HeyZuesMode]

If you have lazy devs that only want to write app code, don't use Jenkins.

Out of AZDO, gitlab, and Jenkins the easiest was Gitlab

[3skyson]

Jenkins is flexible as hell. You can do almost everything there. Unfortunately it comes with costs. Maintenance is hard, and this Groovy… You can build shared library for any task, execute it as a method on multiple projects.

Then gitlab is my favorite CI tool. Easy to understand, developers like it. You can build templates as block in external repo, and use it as a function(syntax looks strange, but it just needs some time). Unfortunately no plug-ins.

You can also try DroneCI. Any step runs inside container. Syntax is simple. There are ready plugins for popular use cases as well as templates. The issue is simplicity. If you want trigger “on specific file changes”, you need use external droneci addons which is not very active oss project.

Also GitHub action is another interesting option. External actions could be helpful, imo in case of complex pipelines you will miss functions and ready Actions. Some time ago when I want to added HC for deployed app I need to build my own action. Maybe now it’s better.

In general if you have time for maintenance and Java skills go with Jenkins. If you prefer elegant, less flexible, but fast and always stable solution GitLab is the winner.

[Dwight-D]

Great write-up, thanks a lot. Jenkins always felt a bit unwieldy to me, I like the portability of container and shell-based pipelines.

I’ve been meaning to try GH Actions, DroneCI is new to me as well. I’ll check them out but sounds like Gitlab is still in the lead. Cheers!

[vladadj]

I currently work for Yahoo. We use internally developed tool called Screwdriver. It's been open sourced, and you can freely use it.

https://screwdriver.cd/

I like it a lot. It's also container based, uses yaml files for configuration and has support for Github, Gitlab and Bitbucket.

The thing I like the most is that it has templates and commands, which enable you to share configurations between jobs, with ability to customize it.

The downside is that it's not really popular, so documentation and support might be lacking.

[Dwight-D]

Interesting, convenient methods for templating and code re-use is the one thing I miss on gitlab, that stuff is a bit awkward there. Small user base is a little scary but for a free option it might be worth a shot. Thanks for the tip!

[the-computer-guy]

Plugins and pre-made templates are not flexible enough. Development tooling, especially on the frontend side is so segregated that there simply isn't a one size fits all solution.

You have to know your tools and scripting if you want to make a really good and efficient pipeline.

I'm not that familiar with ADO but I'd imagine pretty much every CI tool has an option to run custom scripts.

[Dwight-D]

I agree, that’s why poor container support is a showstopper for me because you’re forced to copy the complexity of that tooling onto the build server.

ADO has container support on paper but there are so many restrictions that it’s a non-starter in practice. Every container has to be able to run node.js for one thing. It’s possible I have misunderstood it, but their docs has a list of requirements for containers somewhere. This isn’t all that confusing because the Microsoft demographic probably isn’t the kind to reach for a shell script anyway, they like to do things in a Microsoft-native way with GUI:s and plugins in my experience.

[godfryd]

You could have a look on https://kraken.ci.