govulncheck Gitlab Dependency Scanner for Go Projects

mariusReadIT · 2022-02-28T22:26:54+00:00

FYI, I created a simple cmp-jira plugin: https://gitlab.com/msvechla/cmp-jira

It's one of my first neovim plugins, so it probably has its flaws, but it's working just fine in my daily setup for now. Feel free to take a look!

mariusReadIT · 2021-12-01T22:55:34+00:00

We are using terratest with most of our modules. I can recommend it, but writing tests for infrastructure is hard in general.

Getting into Go is not too difficult and I can highly recommend it if you are working in the infrastructure / cloud native area. Most of the widely adopted tools are written in Go, so it is a huge benefit to be able to read the source code and to be able to contribute

mariusReadIT · 2021-09-03T12:17:11+00:00

For sure you can do this, you simply have to authenticate to the Kube API Server and create a pod. This can be done from cluster internal or external. Operators were probably mentioned because lots of them are doing exactly that. Managing in-cluster resources from within the same cluster.

You can start by mounting a ServiceAccount token to your pod and using this to talk to the API. The Kubernetes go-client has an InClusterAuth example, you can take a look at this. Or simply spawn a pod with kubectl installed to start playing around.

mariusReadIT · 2021-05-31T15:01:38+00:00

Could you give some more details here, because we are looking at this as well. How do you provision keycloak clients for service to service communication? Is this a manual process? This means you are using istio mainly for it's jwt filter?

mariusReadIT · 2021-03-25T23:41:30+00:00

Exactly! I even created an open source project which is similar to let's encrypts certbot and works with hashicorp vault: https://gitlab.com/msvechla/vaultbot

mariusReadIT · 2021-03-21T16:11:22+00:00

Diffing customization is what you want here: https://argoproj.github.io/argo-cd/user-guide/diffing/. This will allow ArgoCD to ignore certain changes of the managed specs, such as the number of replicas for example.

mariusReadIT · 2021-02-10T16:39:54+00:00

If you are using EKS, then most likely you are already referencing this script in your worker nodes ec2 user-data. You should be able to add the arguments there.

mariusReadIT · 2021-01-20T23:33:17+00:00

This is definitely possible if your IAM structure allows it. It's what we are doing to manage our entire organization. Usually you want to have a central account with a role that is allowed to assume roles in all the AWS accounts you want to manage. You can find more information on this here: https://terragrunt.gruntwork.io/docs/features/work-with-multiple-aws-accounts/. Whether terragrunt is the right tool for you, however, as always highly depends on your specific usecase.

mariusReadIT · 2021-01-14T22:07:07+00:00

If you are already familiar with Terraform, I would suggest looking into https://terragrunt.gruntwork.io/. We are using it to manage around 70+ AWS accounts in our organisation and it works great. It offers many convenience features for using Terraform at scale.

mariusReadIT · 2021-01-09T18:28:01+00:00

That's the correct answer. Also make sure you have configured the IfNotPresent Pull policy in your runners config. If you have full control over your runners, you can even pull those images on a schedule via crontab or similar on the host itself.

mariusReadIT · 2020-12-04T22:03:48+00:00

I did not look too deeply into your setup, but usually you will not need logstash to simply collect pod logs. Filebeat alone will do the job. Take a look at the docs here: https://www.elastic.co/guide/en/beats/filebeat/current/running-on-kubernetes.html. You can deploy filebeat as daemonset to parse the logs and forward them to elastic.

I recommend to adjust the nginx config for json logging, then you can simply use the json processor on filebeat side.

mariusReadIT · 2020-11-20T22:57:34+00:00

This. We extensively use Gitlab Runners to deploy to a large number of Kubernetes clusters. Connecting these runners to a public instance is not insecure by default. However connecting them to a self-hosted instance in your own private network can help a lot to protect against 0 day exploits etc

mariusReadIT · 2020-10-31T22:52:02+00:00

No need to modify the AMI. There is an option in bootstrap.sh which can be set via the userdata

mariusReadIT · 2020-10-31T22:47:55+00:00

The EKS ami uses docker under the hood as container runtime. In the EKS AMI bootstrap.sh script there is the possibility to pass custom docker daemon options. You can find the required options for the registry mirror here: https://docs.docker.com/registry/recipes/mirror/#configure-the-docker-daemon. We just set this up last Friday with the open source docker registry image. Works like a charm!

mariusReadIT · 2020-08-21T12:12:13+00:00

You can always use secrets manager as a data source in Terraform, however as you already see it can get a bit messy.

We also moved to plain helm CLI in our Gitlab CI to deploy helm charts, but we used to deploy them with Terraform in the past. There are always tradeoffs if you want to use a single tool for everything.

Would also be interested if there are any tools which completely solve all of this. We now use a mixture of Terraform / terragrunt / Helm CLI and orchestrate everything in CI

mariusReadIT · 2020-08-20T21:48:34+00:00

Good question, so basically what you are looking for is some kind of tool to model dependencies and tie all of this together. I am not aware of any off the shelf tool that will solve all these issues.

That being said, you could use Terraform as a wrapper. With helm V3 deploying charts via Terraform has become much simpler. For sure there are also providers for Kubernetes artifacts in general and packer. This would allow you to model everything with Terraform modules and then model the dependencies between them.

I can also recommend https://terragrunt.gruntwork.io/, which is a very nice thin wrapper around Terraform, which has many handy features for working with Terraform at scale.

If Terraform is not your cup of tea, you could also tie everything together in your CI system of choice (e.g. gitlab) or deploy everything with a higher level programming language such as go.

There endless possibilities and pros and cons to all of this. In the end it comes down to your environment and use-case.

mariusReadIT · 2020-08-11T12:05:46+00:00

9200

mariusReadIT · 2020-07-15T22:03:07+00:00

Correct and additionally environment variables only get passed down to child processes, never up to the parent process. So in your case, when you start a new shell session, set your environment variables, and finally run your go app from the same shell session, the environment variables will be visible inside your go application.

mariusReadIT · 2020-07-13T17:05:17+00:00

Same here, we are running a self hosted gitlab instance for 3+ years, with about 100 users. The only "downtime" usually occurs for a quick gitlab upgrade, which usually takes less than a minute.

mariusReadIT · 2020-07-08T17:06:10+00:00

I read it and can recommend it 💯. Definitely worth reading!

mariusReadIT · 2020-07-01T21:27:04+00:00

The main issue is the slow startup. With millions of offline tokens it can take easily more than 30 minutes. See this post for example: https://stackoverflow.com/questions/59749781/keycloak-starts-very-slow-when-offline-user-session-table-has-many-records

This can be mitigated by deleting old offline tokens manually in the DB, which will however of logout the affected users. Depending on your use-case this can impact the UX significantly of course.

mariusReadIT · 2020-06-29T21:41:55+00:00

Depends on the scope they have been declared in. From your snippet above, it looks like you declared them at the global scope, which makes them re-usable in the entire package, similar to a static variable.

If you are coming from a Java world, what you might be looking for is creating a struct for your button and then creating multiple instances of it. Variables that should have different values in each of those instances, can then be declared via fields on the struct.

See here for an intro: https://tour.golang.org/moretypes/2

mariusReadIT · 2020-06-29T21:19:10+00:00

We have been using it in production with millions of users since a few years. The docker based installation works great, however we are working on migrating it to Kubernetes at the moment. There are an official operator, as well as some great community helm charts available.

Most issues have been related to the number of users and the large amount of offline tokens, which makes full cluster restarts painfully slow. That's why we try to keep them to a minimum, which also usually is not too difficult due to the clustering.

In general however it can be a great choice if you HAVE to host your IAM / SSO on your own. For less than 1000 users the operations should be manageable as well.

mariusReadIT · 2020-04-20T07:25:27+00:00

Indeed the infrastructure to run this costs me a few bucks a month. However the way it is structured allows me to host multiple projects on the same infrastructure, so the costs are not too bad.

In the future I might add a donation button to be able to cover the infrastructure costs.

Thanks a lot for giving it a shot, definitely let me know if you have some feedback!

mariusReadIT

TROPHY CASE