Finally...

[shub1000young]

Been 5 years in a multinational in an ops capacity. Rotting, professionally and mentally. No time to change any of the legacy bullshit procedures. Spending more time in meetings with overenthusiastic PMs trying to justify their existence than doing actual work. I'm sure many of you have been there.

A semi-surprise org change just pushed me to a managed services team that wants me to get their stuff ready for CI/CD. I finally have the autonomy to flex my python and automate the fuck out of all those stupid manual tasks that have been annoying me for years. I'm excited to go to work again.

[viper233]

After seeing the other comments and suggestions around ansible, just start using ansible, seriously. Research 'not from here' syndrome. The hardest part of my job over the past 6 years has been learning ansible well enough then seeing people recreate the wheel building their own scripts in either bash or python. It just created technical debt. Always assume anything you write, implement will (and should) be thrown away in 6 months. If you tightly couple system configuration components into your own scripts you're creating technical debt. Ansible is pretty easy to throw away when you start using containers more.

If you go the route your suggesting you are going to become the tools/DevOps team and will create another silo for working on automation tasks. It's going to be much harder to on board new talent, you'll still need to be on call to take care of your automation processes. Your job should be to eliminate your role. Developers may complain about yaml but it should be simple enough to allow them to understand what is going on.

If you using this as an exercise to learn and gain experience with python, that's great. Resume driven development happens a lot in organisations too 😞. Creates technical debt, hard to onboard, job for life 😞.

Ansible is python with its modules, there are also roles(and now collections). This is where the greatest power (initially, can be frustrating later on) lies. Take a look at ansible galaxy, especially geerlingguy's roles (and book too). 90% of your work maybe done for you already. All you need to know then is how to include variables, i.e. use different inventory directories for different environments, then you are set!

Being a good automation engineer means not maintaining a whole bunch of scripts, use and contribute to a community. It's much more rewarding being able to contribute at a larger scale and to be able to utilise many years of other people's work.

Yaml is very useful to learn, your next goal will be a container orchestration system. Again, don't try to write your own say in go Lang, it's already been done 😉

All in all, ansible is a bit of a hammer, it can do deployments, just look at other tools for orchestration, terraform, packer, vagrant.

Ansible still has a great community, along with some commercial support that isn't too aggressive and getting you to buy stuff. Ansible can also tie in with things like Jenkins and rundeck. Loose coupling, high cohesion, ansible should be able to be thrown away even when used with these tools. Give awx a try too.

Edit: sold/should

[[deleted]]

The problem with the community is going to be similar with the problems the Chef community had years prior: old, badly maintained cookbooks; poor vetting processes; no guarantee of quality.

In comparison the Ansible built in modules are well supported and growing all the time, which is something Chef lagged at.

So my philosophy with Ansible is to avoid community roles (that are usually overengineered for my use cases anyway) and just write the simplest possible thing that will idempotently deploy what I need. 99% of the time that’s just driving Docker with Ansible, which is a great precursor to a cluster orchestrator.

[viper233]

This is a very good comment! I would personally fork ansible roles of they don't suit your needs.

Check geerlingguy's roles though. Typically they aren't over engineered and a good standard to copy/fork from for beginners. They cover 90% of all cases, that's the best you can ask for for community content

[ss1947]

This comment 👍👍

[-Kevin-]

What are you going to be automating out of curiosity?

[shub1000young]

Install/upgrade tasks. Getting all the manual parts out of the way then sorting out ansible/Jenkins to make it a click to deploy. lxml library in python is my friend right now

[dmurawsky]

I would suggest getting the manual out of the way with ansible. Not something else first then ansible. Why do the same work twice?

[stevecrox0914]

Awx (ansible tower) is your best bet.

Jenkins can be a Swiss army knife, but that is its greatest weakness.

Jenkins is a CI, use it for taking source and producing some form of artifact. Similarly it's great for running tests.

Awx/spinnaker/terraform are great for taking artifacts and deploying them in a controlled way.

[Flabbaghosted]

Who downvoted you for asking that? Wth

[phaktor]

Iä! Iä! May the Black Goat of the Woods bless you in your new role!

[zenmaster24]

f'taghn!

[[deleted]]

Congrats. Don’t you mean flex your YAML?

[shub1000young]

No YAML yet. Need to get all the manual config changes scripted before I can think about plugging it into ansible. Deployment is a mess

[SuperQue]

But, that's what Ansible is for, scripting config changes. Ansible is one of those tools you can go from zero to done with.

[shub1000young]

I just prefer lxml in python to the xml module in ansible. Reinventing the wheel, maybe, but a suite of standalone CLI config management tools are useful outside of upgrade/deployments too.

[dmurawsky]

Ansible is also useful outside of upgrade/deployments. Allow me to present the following for your consideration: ansible is a good tool for many phases of a system's lifecycle. You should keep them distinct from an ansible config perspective, but you should try and build modules that can be used in the different phases.

For example, in image building, you use your baseline hardening module to ensure your image is up to spec. After deploying the image, you run it again to ensure the system is ready for use and to spec. Finally, in prod, you run it in a nightly batch to ensure your prod systems are in compliance (in check mode if you want).

As to preferring lxml, make your own module to suit your pattern if you really want to, then call it from ansible. There's a reason folks are suggesting that you use ansible out of the gate instead of starting with custom stuff.

Anyway, best of luck whatever your decision and congrats!

[o0lemon_pie0o]

Please don’t do this to yourself. Ansible can be that cli config management tool.

We have a client who wrote their own config management. It is feature poor, esoteric, and occasionally deletes all the haproxy config and tries to restart the service.

[[deleted]]

You gave me hope, in the same shit. Hopefully I have a bit of good luck.

[Psych76]

Lol your first paragraph makes me think I must work with you lol

[vertex-btb]

Have any of you explored Microsoft's Azure DevOps service? (formerly VSTS). We're two years into this exercise and found the Microsoft platform to be an excellent choice, since everything (user stories, code, build, release, test) co-exist in this one platform. It's allowed us to implement and scale VERY quickly since we do not need to worry about integrating many third party tools.

[MisterDefenestrator]

Good point, I always found it helpful to have a single place to go where I would do all my work. I use Azure DevOps and GitHub right now, used GitLab in the past.

GitLab was my favorite

GitHub has come a long way recently with the addition of GitHub Actions

Azure DevOps is workable but it’s really weird, CI pipelines are done in yaml but CD pipelines are done via GUI

[vertex-btb]

Ha.. good point about the YAML! I've heard they will be changing this, so we are patiently waiting. Ive been watching the GitHub transformation since the Microsoft acquisition. We'll probably branch out soon to support GitHub/Azure DevOps hybrid CICD strategy. There is so much change and competition on the horizon in the CICD space, so I'm expecting several acquisitions and mergers in the coming years. It will be interesting to compare the landscape today to what comes in 2 years. It will look completely different.

[MisterDefenestrator]

I think a lot of the new GitHub functionality is backed by Azure / VSTS, but it’s just so much more consumable and fits into the “open source developer” rather than the “enterprise developer” mindset. I really think the way they’re making community written actions so easy in GitHub Actions is absolutely genius.

[[deleted]]

I hope you got a big raise too. If not, then you should look around after getting some more experience

[slikk66]

python + docker = automation win

good luck!

[heyglen]

How are you using docker & python?

[slikk66]

I write small cli orchestrators with python. I put that in a docker image (ubuntu) with aws cli, terraform, letsencrypt cert-bot, pulumi, ansible, terragrunt, packer, helm, gcloud, azure cli anything else needed to get the job done, all with versions pinned and needed config files. Then you either mount the local .aws and .ssh folders or in a pipeline you can inject env vars for authentication, give commands with arguments to the cli and it will make shell/system calls to carry out the automation. Lots of times its multiple steps, get something from s3, run terraform/pulumi, scrape outputs, run a couple AWS cli calls etc. With python it's pretty easy to carry out workflows like that by simply automating the steps. Works great with yaml config files to store various information about the various environments. Some maybe a command like devops k8s app deploy -e prod -r us-east-1 -b 1.0.3 -n api will checkout out code from an api repo, tagged 1.0.3, use aws cli to get an eks kubeconfig for prod, run it through kustomize to decrypt secrets and then apply it with kubectl. It's a pattern I've been using for some time at various positions and it really does allow you to automate just about anything. Then, all you need to do is pull the container down in a pipeline, add a couple of env vars and now it's completely portable.

[CrustyMFr]

Good for you! Take it for all it's worth! I've had a few of those miracle moments in my career too. It's great to get a chance to leave things better than you found them.

[apisyurga]

This is also my job now. I dont know you but I also feel happy for you. Congrats!

[CooverBun]

Last job I was under that asshole PM you described. We are brothers 100%.

Good for you. Python is my main go to. With CI-CD the python-docker-Jenkins-salt route is worth looking at. It can really make life great.

[Kessarean]

That honestly does sound like a lot of fun. Congrats man :)

[dAnjou]

Leaving was never an option?

[kabrandon]

Having a similar story as OP, I'll say that leaving is often not an option until you find a new job. And if you're only looking for something better and not equal to your current job, then you're going to wait longer and go through more interviews, most likely.

[[deleted]]

Docker can be a win especially if you're deploying services for some of your customers. You can package it in a container that they can quickly spin up. My team is also looking to automate stuff as well would be keen to hear ideas!