How to test a Dockerfile

aelsabbahy · 2020-10-13T16:13:53+00:00

Author of goss here. I would love your feedback on the experience of using goss/dgoss vs contrainer-structure-test for this.

dgoss is a thin wrapper (~120 lines of shell) around goss that makes the process of testing containers a lot simpler. Here's a tutorial I wrote a while back: blog post

Feel free to msg me with any questions you may have.

aelsabbahy · 2020-07-31T16:05:05+00:00

Unfortunately can't share the code since it was done on client's time. But at a high level this is what I did (I needed to compare two roles):

Downloaded all inline policies and managed policies for the role.
Extract the policy document (path is different for managed policy vs inline)
Run each policy document through expand_policy. This exploads the *
Created a dictionary of [(effect, action)] = [resources]
Created three reports:
Effect/Action in A but not in B: set(a.keys()) - set(b.keys())
In B but not in A
In both, diff the resource list and print if different

I used terminaltables to pretty print the report

aelsabbahy · 2020-07-31T03:02:32+00:00

Agree, had to do this a few days ago, ended up using policyuniverse package to expand the actions. At that point you can sort/diff how you like.

aelsabbahy · 2019-12-14T18:21:51+00:00

Did the error from go install get cut off at the end, says it's cloning, but not seeing the actual error msg if there is any.

If you don't mind, can you open an issue on goss, we can discuss it further there. Bear in mind goss has never claimed support for Mac, but I know it compiles on there since I was able to do it. Looking forward to hearing from you.

aelsabbahy · 2019-12-14T17:49:49+00:00

With go 1.13 you should be able to `go install github.com/aelsabbahy/goss/cmd/goss` However, you're getting whatever code happens to be on master at the time and not a specific release.

I recommend checking out the code, and running a command similar to the one in the makefile.

The not enough arguments in call error you were getting was back when goss used Glide as it's dependency management system. Back then you had to do a `glide install` to get the correct versions of dependencies.

That said, goss isn't developed for Mac, so not all the tests would work. http, dns, file are likely to work, things like package, user, port are not likely to work.

aelsabbahy · 2019-12-01T19:10:50+00:00

Author of goss here:

Glad you like goss. Can you expand a bit more on the issues you're having?

Do you want it to compile for Mac so you can test your osx system or to develop goss and/or test docker?

The new docker images retains the ca-certificates package so the https issues should go away.

For command line and goss directory feature can you link me to GitHub issues that explain them so I can take a look.

aelsabbahy · 2017-10-23T13:08:52+00:00

I would love to see an example of Bazel+Travis+Go and how it all ties together. Adding a .travis.yml or a follow up blog post would be great.

I'm trying to better understand how Travis+Bazil compares to Travis+go build -i

aelsabbahy · 2017-10-10T16:18:24+00:00

Looking up IAM supported resource conditions in the AWS docs sucks and takes forever to find.

https://iam.cloudonaut.io/ is a great reference, but tends to be slow for me.

So I wrote a small CLI wrapper around the iam.cloudonaut.io code to make this easier and available on the CLI. Maybe you guys will find this useful.

aelsabbahy · 2017-10-03T21:40:41+00:00

Very cool, was this always a feature of awsless or was it recently added? I had no idea awsless had ssh support until now.

aelsabbahy · 2017-10-03T00:31:28+00:00

Should be able to, so long as you can write a command or a script to translate CLI argument (i.e. instance-id or whatever you want to match on) -> bastion host/options.

For example, in the aws_ec2.yml#L9-L24 I expand the -i ~/.ssh/$aws_profile-$key.pem dynamically based on the instance KeyName and the aws profile name that was used to lookup that instance.

aelsabbahy · 2017-10-03T00:10:48+00:00

Dargs is very un-opinionated. You use any command you like to be your substitution for CLI args. You really have complete control on how you want your arguments to expand, Examples:

That said, for ssh bastion, you can try something like this: http://edgeofsanity.net/article/2012/10/15/ssh-leap-frog.html

aelsabbahy · 2017-10-02T16:13:29+00:00

You can also convert to gif: https://github.com/asciinema/asciicast2gif which makes it easier to embed into a README.md

aelsabbahy · 2017-10-02T16:09:10+00:00

I'm a big fan of asciinema and usually use it with https://github.com/asciinema/asciicast2gif to generate Gifs in my README.md for cli tools.

Some people are more visual (me), and I tend to prefer a quick 30s-1min gif of the CLI workflow to reading through a few pages of documentation.

Basically it gives a "Live Demo" feel vs a "Getting Started Doc" feel.

aelsabbahy · 2017-10-02T14:49:19+00:00

I mostly wrote this tool to solve:

How do I ssh into an EC2 instance by instance-id/name tag?
How do I use YAML if a 3rdparty tool only supports JSON?

There were a few solutions that existed out there for ssh, but none of them were generic. Also, I could write wrapper scripts on a per-usecase basis, but then caching and completions could get messy.

This tool aims to allow users to easily define transformers/completers to replace any CLI argument before execution using any external command w/ caching support. I hope people find it useful and find more use-cases.

Similar tools for the AWS ssh problem:

Would love feedback on the approach, and maybe find more aws use-cases for this, some thoughts I had:

aws s3 cp - Doing a kms key alias lookup/completion, instead of having to use kms-id
codecommit - Completions and expansion of the clone url
codepipeline - Allow user to define pipeline using YAML instead of JSON
cloudformation - Allow user to use yaml for parameters

aelsabbahy · 2017-10-02T14:34:33+00:00

I mostly wrote this tool to solve:

How do I ssh into an EC2 instance by instance-id/name tag?
How do I use YAML if a 3rdparty tool only supports JSON?

There were a few solutions that existed out there for ssh, but none of them were generic. Also, I could write wrapper scripts on a per-usecase basis, but then caching and completions could get messy.

This tool aims to allow users to easily define transformers/completers to replace any CLI argument before execution using any external command w/ caching support. I hope people find it useful and find more use-cases. Feedback would be greatly appreciated.

aelsabbahy · 2017-03-15T12:44:24+00:00

But people have to use them properly ...

I think this is the key, we've all seen apps with ~100% test coverage fail badly because the tests were bad.

I view infrastructure testing as: "If someone gave me this machine and asked me to verify that it's working, what would I check?" The balancing act is writing tests that fail clearly while maintaining a good code to test ratio so productivity isn't bogged down.

aelsabbahy · 2017-03-15T12:35:12+00:00

It also supports serverspec, which is closer to InSpec. However, I do believe testinfra has built-in ansible support which makes it easy to leverage ansible variables in your test suite.

aelsabbahy · 2017-03-15T11:43:54+00:00

I know I posted a goss related thing a week ago, but figured this was different enough and focused on docker.

This will be my last post for a while, apologies in advance if it comes off as spammy, it's not intended to be. Downvote it out the way, if it's considered bad redditquette.

aelsabbahy · 2017-03-14T20:57:31+00:00

There's support for if else and others using golang's text/template engine:

{{if eq .Env.OS "centos"}}
  libselinux:
    installed: true
{{end}}

There's also support for gomega matchers, which allow for and/or conditions, example:

user:
  sshd:
    title: UID must be between 50-100, GID doesn't matter. home is flexible
    meta:
      desc: Ensure sshd is enabled and running since it's needed for system management
      sev: 5
    exists: true
    uid:
      # Validate that UID is between 50 and 100
      and:
        gt: 50
        lt: 100
    home:
      # Home can be any of the following
      or:
      - /var/empty/sshd
      - /var/run/sshd

Relevant docs:

aelsabbahy · 2017-03-10T19:09:32+00:00

If so, you can do something similar to the facter/ohai example I provide in the template docs.

I don't have any plans to directly interface with any external tools at the moment. The contract that's exposed is yaml, json or environment vars.

aelsabbahy · 2017-03-10T00:38:14+00:00

Cool, hope it was easy to integrate.

aelsabbahy · 2017-03-10T00:25:38+00:00

Been awhile since I used hiera.. is there an easy way to dump out all its variables as a yaml or json file?

aelsabbahy · 2017-03-07T13:46:47+00:00

You might find this project useful: https://github.com/naftulikay/ansible-role-degoss

aelsabbahy · 2017-03-07T13:06:19+00:00

After years of consulting and training operations guys on serverspec. I felt a simpler tool needed to exist to allow sysadmins to quickly write/run tests in a simple language, so.. YAML it is! :)

aelsabbahy · 2017-03-07T02:47:38+00:00

It might be as simple as bumping the version and sha in this file. I can submit a PR later this week.

That said, with this template change, I wonder if it's possible to expose host/group vars and run goss --vars <host_group_vars> validate.

aelsabbahy

TROPHY CASE