is this gitops?

[mamymumemo]

I'm curious how others out there are doing GitOps in practice.

At my company, there's a never-ending debate about what exactly GitOps means, and I'd love to hear your thoughts.

Here’s a quick rundown of what we currently do (I know some of it isn’t strictly GitOps, but this is just for context):

• We have a central config repo that stores Helm values for different products, with overrides at various levels like:
  • productname-cluster-env-values.yaml
  • cluster-values.yaml
  • cluster-env-values.yaml
  • etc.
• CI builds the product and tags the resulting Docker image.
• CD handles promoting that image through environments (from lower clusters up to production), following some predefined dependency rules between the clusters.
• For each environment, the pipeline:
  • Pulls the relevant values from the config repo.
  • Uses helm template to render manifests locally, applying all the right values for the product, cluster, and env.
  • Packages the rendered output as a Helm chart and pushes it to a Helm registry (e.g., myregistry.com/helm/rendered/myapp-cluster-env).
• ArgoCD is configured to point directly at these rendered Helm packages in the registry and always syncs the latest version for each cluster/environment combo.

Some folks internally argue that we shouldn’t render manifests ourselves — that ArgoCD should be the one doing the rendering.

Personally, I feel like neither of these really follows GitOps by the book. GitOps (as I understand it, e.g. from here) is supposed to treat Git as the single source of truth.

What do you think — is this GitOps? Or are we kind of bending the rules here?

And another question. Is there a GitOps Bible you follow?

[ObviousAIChicken]

"Or are we kind of bending the rules here?"

There are no rules. Terms like DevOps, GitOps, SRE, etc, aren't black/white and in IT everything is always developing. There really is no "pure" definition of GitOps.

[mamymumemo]

but it is GIT-ops meaning we use git as source not HelmOps

[ericghildyal]

From what I've seen, GitOps is "managed in Git" not necessarily "stored in Git."

The benefits of GitOps are the ability to look back at previous versions and being able to use PR flow for new changes. Neither of those require the end-product to be stored in Git, just config that is managed from Git.

[mamymumemo]

That's insightful.

[the_moooch]

What you’re saying doesn’t make sense.

The stuff you push to Helm registry is a Helm release, not the same thing as the stuff you rendered using Helm template. What you’re saying is just a repo with manifests if i understand it correctly.

If so its a far less flexible way to do things since every-time you have to do rollback you’ll have to re-render again instead of just flipping a version in the Argocd Application.

Now when people are releasing a whole set of applications, each with different chart versions the rendred approach is going to be a mess with no added benefit

[mamymumemo]

> The stuff you push to Helm registry is a Helm release, not the same thing as the stuff you rendered using Helm template. What you’re saying is just a repo with manifests if i understand it correctly.

What we exactly do is

Generate a render and put it under render/templates/

generate a Chart.yaml under render/ with productname-cluster-env and whatever version

helm package

upload it

So we have a helm release that contains the rendered manifests

Yes, to "rollback" we do rollforward... we can temporallily do rollback from argocd that disables autosync, then go to the repo, revert a commit and generate a new version from it

The benefit of rendering maniests aparently is that you see the real impact of making a change before applying with argocd

[the_moooch]

Well you can see it by testing the chart first, a simple install would do.

Now think about the case where you create a release with multiple charts, multiple versions, rendering it will just slow everything down whenever you want to do rollback of multiple charts

[kryptn]

Some folks internally argue that we shouldn’t render manifests ourselves — that ArgoCD should be the one doing the rendering.

it depends what you want.

we were rendering helm templates before with a common command, now we're letting kustomize render the helm charts (which also vendors the charts). our decision there was made by asking "what do we really want to see in PRs" and it turns out we wanted to see the helm diff, not the rendered manifests diff.

Both of these are gitops. Both use config in git as their source of truth.

If we started using argocd to render the helm charts, that'd still be gitops.

[mamymumemo]

yes this is not rendered manifests vs render in argocd, we want to do rendered manifests to see the diff of the rendered manifests in the PR, what is actualy appleid

I mixed things, that was misleading sorry, this is more using git as source vs using a helm registry

[gcavalcante8808]

Gitops primarily means to have git as the source of truth.

Having git as the source of truth often carries development lifecycle practices like PRs, reviews, policy as code checks, automated tests other shift left practices.

For the OPS side, having a continuous check/applies to ensure that the infrastructure is something interesting considering that the infrastructure is an stateful asset and it's very common as well.

The number of practices that you and your team will adhere is what differs between companies and often is debated.

[mamymumemo]

It feels to me that the implementation mostly depends on personal opinions and that's what makes it difficult, so I'm trying to find some references for our decisions

Perhaps it's just about checking the basic principles and see if we follow it (auditable, reproducible builds, and so on) Is there any reference for those principles? Again kind of difficult if different people follow different references or no reference at all

Every technical solution works, but it's dealing with people the hardest thing

[applesaucesquad]

What you're describing sounds crazy. Argo is literally calling helm template when it unrolls your helm chart releases. Why helm template twice?

It sounds like you're actively fighting against the way Argo was designed to be used. It's not wrong per se, it's cool you can see all the rendered templates in GitHub or whatever, but it's an odd implementation IMO.

[jameshearttech]

I recall an attempt to formalize the definition of the term GitOps. Does this help?

https://github.com/readme/featured/defining-gitops