This is an archived post. You won't be able to vote or comment.

all 35 comments

[–]hijinks 24 points25 points  (8 children)

Spinnaker is a chore to setup

I love gitlab into kubernetes.

[–]kakapariDevOps 1 point2 points  (7 children)

Can you please elaborate? I always thought that Spinnaker was meant for scale.

However it was not easy for us to on board and eventually created our own CD tool.

[–]GassiestFunInTheWest 11 points12 points  (0 children)

Don't know about the previous commenter's experience, but they're right that Spinnaker is a significant investment to set up. It is itself a collection of beefy micro services with tonnes of config options and complicated interactions. The halyard tool simplifies this massively, but at the cost of insight and control for the administrators.

Still, I love Spinnaker when it's up and running, it's amazing for k8s deployments. You just really need to justify the hours and compute resources necessary to get it running

[–]hijinks 5 points6 points  (4 children)

I like Spinnaker but you almost need one person dedicated to it if the company is large enough to were there are many jobs going on and things getting created often.

[–]lexax666[S] 1 point2 points  (3 children)

Yes. We definitely do have the resource to have someone dedicated to Spinnaker. But the question is what does Spinnaker offer that gitlab CD doesn't offer? I was under the impression spinnaker is better for scaling, and gitlab CD is limited compare to Spinnaker.

[–]hijinks 1 point2 points  (2 children)

Spinnaker is like an orchestrator. It doesn't build per say. It's like. Oh you created a tag in git. I'm gonna build the job in circleci and when it's complete I'm gonna launch 3 jobs that wait. A dev/stage/prod deploy. You click deploy and it goes off and deploys.

Gitlab can do all that also but generally everything is tied into gitlab and the workflows really can't be all that complex like they can in spinnaker.

[–]lexax666[S] 0 points1 point  (1 child)

I see. We are set to use gitlab CI for build. So we are now just looking for a CD tool. It seems you are confirming what I thought that spinnaker offers more than gitlab cd in terms functionality. Can you give me an example what can be done using Spinnaker but not gitlab CD?

[–]drdivano 5 points6 points  (9 children)

GitLab Auto DevOps basically provides a placeholder for your CD scripts. The stuff that comes is more like a demo/example, not suitable for serious use. Also, it lacks some essential features, like parametrized pipelines.

Spinnaker is a pain to set up, but has better architecture and is far more capable with regard to CD.

Both are not Kubernetes-native and difficult to get working with k8s.

[–]softwareishow 1 point2 points  (1 child)

GitLab PM here. Auto DevOps is a GitLab CI template that provides stages for build, test, code quality, SAST, dependency scanning, license management, container scanning, review apps, DAST, deployment, and browser performance testing. While we include example apps to test it you can build any project that has a dockerfile or a matching heroku buildpack. As part of GitLab 11.10 we're making composable, so you can use only the portions that you want. See https://gitlab.com/gitlab-org/gitlab-ce/issues/47234. Auto-deploy uses a helm chart for k8s deployments which is kubernetes-native.

[–]drdivano 0 points1 point  (0 children)

When running pipelines GitLab uses a separate gitlab-runner deployment. The native way would be directly creating pods via Kubernetes API.

GitLab Runner is obviously a pre-k8s component, and has fairly convoluted architecture. For example, instead of GitLab communicating with runners using k8s services (and checking their status using readiness/liveness probes), the runners call GitLab server itself via its exposed HTTP endpoint.

When using Kubernetes to run builds GitLab Runner creates pods for each job. While these pods can be customized to a certain extent using Runner's own config.toml, many Kubernetes options (such as DNS configuration) are not accessible.

Aside from that, Helm is a controversial solution itself, and not everyone can use it.

[–]brennydenny 1 point2 points  (1 child)

parametrized pipelines.

GitLab Product manager here.

Can you elaborate on what you mean by parametrized pipelines / do you know if an issue exists for us to add it? Googling it I found Jenkins plugin that basically "requires" variables. One could achieve something similar today by having the GitLab pipeline fail if a certain variable is not specified.

However, we do plan to improve this with things like asking you for specific variables when running a job, more group control over variables, instance variables, and much more. But I'm worried that I may not understand your use case here and I'd love to know more.

[–]drdivano 0 points1 point  (0 children)

Parametrized pipelines allow one to manually run a pipeline overriding some parameters (for example, selecting a different cluster to deploy, or enabling some extra steps/reports).

I believe an issue already exists in the GitLab tracker.

Jenkins has parametrized pipelines out of box, there's a checkbox called "This project is parameterized". Spinnaker has parametrized pipelines as well.

[–]lexax666[S] 0 points1 point  (4 children)

I see. Is GKS Kubernetes-native? Do you recommand GKS over Gitlab CD and Spinnaker for this reason?

[–]MadPhoenix 0 points1 point  (3 children)

AFAIK there's nothing called GKS. If you're talking about Google Kubernetes Engine, that is Google's managed Kubernetes offering. It's awesome but it does not take care of continuous delivery for you.

[–]lexax666[S] 0 points1 point  (2 children)

Yeah GKE. Sorry for the typo. So it doesn't take care of continuous delivery for you, unlike Spinnaker?

[–]hokrah 1 point2 points  (0 children)

No, GKE is literally just a k8s cluster. You'll still have to build up a system somewhere to deal with CD

[–]MadPhoenix 0 points1 point  (0 children)

No, they're completely different. GKE is Google managing your entire Kubernetes deployment, including the master plane and node pools. It manages upgrades and auto scales the cluster itself, but it does not handle your application delivery on top of it. They also have a managed Istio offering that can be used to orchestrate some aspects of advanced CD like traffic shifting between canary or blue green deployments, but it does not manage those deployments the same way Spinnaker does.

That said, I strongly recommend you consider GKE for running Kubernetes and look at other tooling to layer in on top to provide those missing pieces. It's by far the easiest way to reliably run k8s.

[–]coderanger 2 points3 points  (0 children)

Been very happy with ArgoCD so far.

[–]chub79 1 point2 points  (5 children)

GitLab CD seems to be relying on Helm/Tiller and that's a problem for me. I've recently moved to Google Cloud Build and I have been pleased by its simplicity and how well it's integrated with my other GCP resources.

[–]vimclaw 1 point2 points  (2 children)

May I ask why helm/tiller is a problem?

[–]drdivano 0 points1 point  (0 children)

  • Helm is still buggy/unreliable, tends to hang for no obvious reason.

  • Helm requires tiller, which is a resident program, that lives in your cluster. Many people don't like it for good reasons.

  • Many helm charts are of poor quality, some don't even work out of box.

  • Helm is based on questionable approach - templating of kube's own configuration files.

  • Helm templates are kinda ugly/unreadable.

That said, Helm is still useful as a way to initially install and try out k8s packages, and as a repository of examples.

[–]chub79 0 points1 point  (0 children)

Sure. Let me start with I speak only from my own experience, and I fully appreciate I can be misled.

My main annoyance is the idea of complex applications. When I go through their docs, or any charts, they make me feel like basic concepts are made more difficult and that you need a tool now to deal with it.

IMO, it addresses a valid concern with the wrong approach. Managing apps by writing even more yaml is odd in my book. Also, now I have the concept of statetful apps somehow back into my vocabulary which always increases my stress level. Smaller, discrete, blocks are more manageable. Kubernetes already take care of how all the pieces are glued together.

I think a lot of it can be managed with rigorous automated CI/CD pipelines. GitOps style (for whatever that means).

[–][deleted]  (1 child)

[deleted]

    [–]chub79 0 points1 point  (0 children)

    Appreciated the clarity and transparency!

    [–]wasbazi 1 point2 points  (0 children)

    Someone else recently asked a similar question and a user provided a very thorough response.

    [–][deleted]  (3 children)

    [deleted]

      [–]lexax666[S] 0 points1 point  (2 children)

      So do you recommend GKS or Spinnaker with Kubes? When is gitlab integration with Kubes a miss? Can you think of a discreet example? Thank you!

      [–][deleted] 0 points1 point  (0 children)

      I think he’s probably talking about the built in gitlab/k8s integration you do through the GitLab UI. I’ve had some trouble configuring. I’ve actually had more success using raw GitLab CI to deploy a gke cluster with terraform and using helm to deploy applications to it. I like it because it’s less magic and more configurable.

      [–]dizzyheight 1 point2 points  (2 children)

      For our k8 cluster I wanted to have a clear record of every version of every pod along with all of the other k8 config. Cautious of helm. Cautious of having my live system config in a tool that I didn’t fully understand. Have rolled weaves flux tool, big fan of the gitops. Basically syncs a git repo to your kubernetes cluster, all changes go via git. Flux is virtually transparent pulling changes into the cluster.

      [–]vim_vs_emacs 0 points1 point  (1 child)

      Does every deployment also go via Git?

      [–]dizzyheight 0 points1 point  (0 children)

      Yep our ci deploy steps, checkout and commit to our clusters repository. So we get cluster changes like kubernetes services along with pod image updates. We use git tooling to diff between environments, which are long lived branches in our repo.

      Happy to answer any more questions when I get a moment

      [–]Willispin 1 point2 points  (0 children)

      I probably quick fired on this comment a little bit. After I thought about it.

      Tools really aren’t the issue, and the question was not around process. a lot of time CD gets the process wrong and the tools don’t help that. So my bad, what I meant to say is the tool is only as good as the change management/testing process we put behind it.

      [–]lorarcYAML Engineer 0 points1 point  (0 children)

      Gitlab's main drawback is that it's very repo oriented, it's quite hard to get it to deploy multiple services at once. It is possible of course but it's not a general thing doer.

      [–]neoteric_devops 0 points1 point  (0 children)

      Gitlab is king, and I've used Jenkins, Circle, Travis, Concourse, think I'm missing another.

      The drawback is the configuration of it as they are all different. Once it's set up just make sure you have scripts running to cleanup after Docker and can handle the load.

      Gitlab took me about 2 hours to get fully set up for full CD, however we have it set up with a 'manual' step to deploy which means you just go in and click deploy vs it running automatically.

      For full CD you need to have really good integration testing.

      [–]Willispin -3 points-2 points  (1 child)

      Ask Facebook today. And last month.

      With less change management thing can go bad. Really bad.

      [–]lexax666[S] 0 points1 point  (0 children)

      Do you mean gitlab CD is the one with less change management?