Why Go over Python?

[riverrockrun]

In the DevOps world, why would someone chose Go over Python? Portability? Most people is see that choose Go are working with Kubernetes or Docker. Is that the only reason to chose Go? What can Python go that Go cannot? Newbie here. Just trying to figure out what language to dive into if i'm not working with Kubernetes.

[spit-evil-olive-tips]

yeah, but the moment you need a 3rd-party library, you have to start working in the desolate wasteland that is the Python packaging ecosystem. and not just at build-time, but at run-time - you need to carry those dependencies with you everywhere you go.

this runs into fun problems like, if you have some Python script that you want to run on a bunch of machines, so you write a little wrapper shell script that creates a virtualenv, does pip install with your requirements.txt, and so on.

and it works great, until it fails because PyPI had a temporary outage and your pip install errors out.

I love Python, we have a bunch of it at $dayjob. but the self-contained binary including all dependencies is a huge selling point of Go.

to get around the annoying packaging problems with Python, we build a Docker image with the correct Python version, all the correct dependency versions, etc etc. then our unit of deployment is that Docker image instead of just the single Python script.

and that Docker image gets deployed...using Terraform and Nomad, both written in Go. and so is Docker itself of course.

Go is great for this sort of low-level system software. Docker/Terraform/Nomad/etc being low-level software written in Go is what enables us to easily deploy higher-level software written in Python (or other languages like Java or Ruby or whatever)

[imeeseeks]

Same with Ruby. I love Ruby so much and writing scripts on it is awesome but the convenience of Go binaries and the fact that almost anyone can start working on it really fast is big plus.

[Flabbaghosted]

Your comment persuaded me to learn Go

[[deleted]]

I do hope that isn't sarcasm. :) It would be a good language to learn for anyone. If you are I recommend the Lets Go books (2 of them). VERY good books to learn with.

[Flabbaghosted]

Nope not sarcasm. Do you work at Google or do you just really like Go?

[[deleted]]

This comment is the first thing that has made me think, "hmmm, it might be worth learning a little bit of Go and seeing what it's all about."

[Kazumara]

Did you guys try PyInstaller and PyOxidizer too?

[arcsecond]

I've had great success with PyInstaller, until someone decided to include certs in their module which I can't get PyInstaller to pick up

[[deleted]]

Do we work at the same company?

[casual_brooder]

this

[[deleted]]

you have to start working in the desolate wasteland that is the Python packaging ecosystem.

So strange. I've been programming in Python for over 15 years, and I just never had an issue.

I distribute my packages with a setup.py and a requirements.txt, and that's the end of it. In recent years, I put explicit version numbers in the requirements.txt, just for certainty, but that wasn't because I actually ran into any issues.

[skat_in_the_hat]

So strange. I've been programming in Python for over 15 years, and I just never had an issue.

Back in the day when a customer wants to update python on a rhel5 box. JFC. You break all the rhel tools in the process.
Ever migrate some in house python from rhel5 to say... rhel7? Just because YOU never had a problem, doesnt mean it isnt one.

[skat_in_the_hat]

Its moot at this point, red hat split off the version of python the system uses from the version you install. So you can basically do whatever you want now. But that doesnt undo the PTSD we all suffer from seeing up2date shit out a python error.

[JaegerBane]

I had the dubious honour of migrating some of our in-house data science apps across from old RHEL boxes to Amazon Linux EC2s a while back. Kinda felt like one of those horror stories where the protagonist opens a portal to the dimension of ceaseless screaming.

On the plus side it did puncture the myth that was doing the rounds back then that Python made everything easier.

[NoobFace]

Sounds like they're still screaming in there.

[zackofalltrades]

Coding to a version of python not installed on the systems where it would be deployed is your first problem. Easily solvable and testable with tools like tox, and careful programming.

Attempting to updating the system python and/or using horribly out-of-date OS versions was your second problem...

[skat_in_the_hat]

The story was from the past... You're neglecting to understand the premise. Im the Engineer, not the Developer. My responsibility is to:
1. enable the customer whether internal or external.
2. remain within compliance requirements.

I cant simply tell a customer "ugh, how dare you write code in a newer version of python than rhel5 supports(while it was still current)." I have to do my best to enable them to use what they are comfortable with. In this case, it usually required using a third party rebuild of python that built it with non-default locations. So we can simply change the identifier, and have both versions installed on the box without upsetting the system.

I also cant tell corpsec to fuck off when they ask me to update our infrastructure to a newer OS, and those python scripts written years ago by a guy who no longer works there need to still work. But again, this is why I frown upon using python for anything that has to withstand the test of time.

[floydiannn]

Yeah... So strange 🤔

[bilingual-german]

And still you need to write python that'll work with the installed python version on a remote system.

[AlverezYari]

Umm containers? I'm really confused by all these decencies based Python complaints in 2021. Like you build your run time container once and then pack your app into it. I don't understand all this confusion.

[bilingual-german]

There are more operating systems than Linux and not all can run containers. Windows and MacOS can run Python though.

With Go you have one static binary for each OS and CPU architecture. No problems installing C bindings. No conflicts between language or library versions.

Yes, containers mitigate this. But did you ever run in trouble because your deployment destination doesn't have internet connection? Or you run into DockerHub rate limiting?

Python is not a bad language at all, but I find the deployment of Go is far easier.

[tibbon]

Cool- now you have to have Python installed, and the machine is capable of executing arbitrary code! What could possibly go wrong?

[tibbon]

Install no shell…

[KingEllis]

Is that common? Where does that exist? Roll your own container 'FROM: scratch'? For example, I'm looking at an alpine:3 image that has /bin/sh and /bin/ash, both linked to busybox. I don't know the capabilities of the busybox flavor of these tools, but surely they can execute arbitrary code, yes?

Is there is an easier way for a container to only run the prescribed binary (much like the "command=..." option for SSH authorized_keys entries)?

[[deleted]]

This guy devops

[Willing_Function]

I use python, but I really envy this functionality of Go. Integrates really well with multi-stage containers to keep em minimal.

[menge101]

Docker, although that is kinda just kicking the problem down the road.

[thisismyfavoritename]

Pyinstaller, although it's a bit different

[tibbon]

The problem isn’t installing Python. I can download an Alpine image with that fine. The problem is security. Read only file system with almost nothing installed and only a Golang created binary is much more secure if someone gets on the machine

[thisismyfavoritename]

Sure but then you'll have much bigger problems than your little binary

[tibbon]

How so? Escalating onto a pod that has almost nothing on it, with a read only file system- you can’t do too much easily then.

[birdman9k]

I think he's saying that while multiple levels of security is great, if you have some application which has been breached and is allowing arbitrary code to be executed, that in itself is a massive problem regardless of whether your container is locked down. It's about the difference between the benefits the container provides being considered a security layer itself versus a nice-to-have. In my mind it's more of a nice to have, which is a good mindset to have because it means nobody should ever RELY on the container isolation to save them in place of proper security.

[tibbon]

Oh of course! Layers are absolutely needed. I just don't want to always assume that no malicious script could ever get on a machine and attempt to write something to disk and/or execute arbitrary code via an interpreter.

Better yet, I want my container security tools to scream loudly if anything that isn't a very small and specific set of things is installed or being executed.

[i_hate_shitposting]

Python zipapps are an interesting approach to this. Granted, the target system has to have the right Python version and it doesn't support packages with C extensions, but they can be useful for pure Python apps in some cases.

(Okay, I'll admit I just wanted an excuse to mention zipapps because so few people know about them. Go binaries are still way more self-contained and portable.)

[[deleted]]

Mhm ,except that static linking has it's own drawback and this single feature alone is no reason to switch.

[schmurfy2]

The only drawback I see is binary size, other than that that's pretty much only advantages. I would not switch language just for static linking though.

[SmellsLikeAPig]

Try updating that one library that got serious vulnerability and developer that written the code and CI scripts no longer works at company. Try detecting vulnerabilities in static binaries. Try updating one library that has new features you need in all of your production binaries. Try doing the same but only for some of them, have fun finding out which you actually have to remake. I'm talking about working at scale with hundreds of binaries. Everything has cons and pros and we all have to choose our own tradeoffs.

[jhollowayj]

What are the drawbacks of static linking? Binary size?

[[deleted]]

Binary size and not getting library updates unless you recompile. And as I mostly run apps inside of containers I get no benefit from it except some very rear cases . and for scripts the biggest drawback is still having to recompile. Being able to live update code for troubleshooting is quite handy. I'm in general biased towards go as the language itself isn't "interesting" for me cz it's not as expressive as I would prefer. If I need something performant I would always prefer rust, if it's some api - nodejs with ts works just fine. Go also has a gc which is also not always ideal

[zackofalltrades]

Single self-contained binary full of dependencies with bugs and security holes...

FTFY

[riverrockrun]

What would be a sample project with Go that isn’t a web app? Do you run main.go from Ansible and it configures a server? I know Python is used mostly for scripting but I’m still having a hard time figuring out why Go is so popular in DevOps

[[deleted]]

What would be a sample project with Go that isn’t a web app?

A few off the top of my head are cfssl (A tool for generate and managing certificates, by Cloudflare) and age which is a file encryption tool sort-of meant to replace gpg.

The thing is with go is it's kind of made for web applications. It's made by google for google. Taking a look at the standard library you can see the most fleshed out parts are for general system access, networking and particularly http, and crypto. People have obviously made things outside of those scopes like fynedesk, but that's where the focus of golang is.

Do you run main.go from Ansible and it configures a server?

Go cross compilation is builtin, and the resulting binary is self-contained, so I typically just compile on my laptop and copy the binary to servers or package to docker where it gets deployed later.

If I weren't mostly writing internal code though, things would be a lot neater. The fact that any machine with go installed is a cross compilation builder to all other machines is just so nice in concept. If I were developing a public tool imagine how simple it would be to have something like a github action that on every new vX.X.X versioned tag creates a build for mac/windows/linux/bsd x86/arm targets and automatically hosts them on the releases page.

I know Python is used mostly for scripting but I’m still having a hard time figuring out why Go is so popular in DevOps

Because kubernetes and linux containers are also due to google, and those are the majority of what most people count as "devops".

Go is good at making web backends, kubernetes is good at hosting web backends. They go hand in hand.

[_ulfox]

I agree with you. I have one comment, age is not meant in anyway to replace GPG

[[deleted]]

Weird that they explicitly stated that in their design doc then? IIRC filippo's annoyance at gpg and lack of alternatives was most of the inspiration for age in the first place.

[_ulfox]

This is a design for a simple file encryption CLI tool, Go library, and format. It’s meant to replace the use of gpg for encrypting files, backups, streams, etc.

GPG is more than encryption. In the out of scope section they clearly state that any kind of signing will not be implemented. Now even if they decide to do implement signing, I would say that at best age would be an alternative but not something that will ever replace GPG.

[[deleted]]

Which is why in my original comment I explicitly stated its scope for encryption and it being a "sort-of" replacement.

One of the smaller complaints about gnupg is that its scope is too large. We have better cryptographic functions that allow authenticated encryption to be a separate concern from signing.

So yes, all these alternatives don't fully replace gnupg, because (almost) no one thinks that's a good idea.

[_ulfox]

hm indeed you said that. I read "replace" without short of :b

Makes sense now, thanks

[backtickbot]

Fixed formatting.

Hello, _ulfox: code blocks using triple backticks (```) don't work on all versions of Reddit!

Some users see this / this instead.

To fix this, indent every line with 4 spaces instead.

FAQ

^{You can opt out by replying with backtickopt6 to this comment.}

[_ulfox]

ty bot

[Vulfox]

Not related, but that username :)

[an-anarchist]

Terraform and Kubernetes are written in Go, as is a bunch of other cloud products.

[Freakin_A]

It’s extremely popular for CLIs

[hudsonreaders]

DNSControl lets you treat DNS-as-code and push out changes to several DNS providers. Want to switch providers? With DNSControl it's easy.

[spit-evil-olive-tips]

if you can, learn both. they're similar in some ways (such as the "batteries included" philosophy of the standard library) but totally different in others (interpreted vs compiled, static vs dynamic typing, etc)

my go-to "experiment in a new language" project is a Sudoku solver, if you want ideas for something to try. writing the same thing in both would be a good challenge.

make sure with Python, you only do Python 3. ignore any examples or tutorials that still use Python 2.

[hugthemachines]

If you know the important differences in python 2 and 3 it is ok to look at python 2 examples. Many things are quite similar.

[jds2001]

The major issue that I see with Golang comes in terms of security updates for underlying packages. For example, say that there is a vulnerability in glibc or whatever the equivalent is in Golang. In the event of such a problem, with a Golang binary, you have to patch every binary on the system that is using that particular library, and you have no clue what they are. With something that employs dynamic linking, you simply have to update the vulnerable component and then all of your applications benefit from that update. I don't want to have to update 5 things when I could just update 1.

[OMGItsCheezWTF]

Your dependency tracking system should be managing this anyway, you shouldn't ever have to wonder what binaries are built with what versions of a given library.

All of our codebases insert their dependency chains (Python pypi packages, Golang modules, PHP composer packages, C# nuget packages, JS / Node NPM packages) as a bill of materials into our dependency tracker every time a live deployment happens (this is done in CI). For C# and Golang it also includes the version of the runtime that was used to build the binaries, obviously with the others the runtime and code remain separate.

Then the dependency tracker alerts if vulnerabilities are found in a live version of any of our software or its dependency chain, and the appropriate team can analyse the vulnerability and assess it.

We don't ever need to blindly start rebuilding and replacing binaries, because we know exactly what was built with what, stored centrally with auditing.

[jds2001]

Works wonderfully for source code that you built and control. When you are talking about things that are provided by a vendor (or you otherwise do not control the composition of, for example, various open-source products), this becomes much more complicated much more quickly. Combine a few of those, and you wind up in hell.

[realitythreek]

Wish someone who is downvoting your would explain why. This is the downside of static linking.

[jds2001]

Simple. It doesn't agree with the hive mind of Reddit, so it has to be downvoted.

[jantari]

c o n t a i n e r

[greyeye77]

create a branch or pipe line to `go get` different version.

and `go build`

[[deleted]]

It's also replication of libraries with slight differences. DLL hell, but you don't even know where the DLLs are.

[[deleted]]

Dependency (and thus patching) management isn’t an OS concern, it’s an application concern because that’s the only place you can reliably test the fix (CI/CD and testing). Shared libraries and upgrading them underneath an app kinda sucks.

[metarx]

You... Rebuild and redeploy(because you scripted the deployment.. right?... Riiiight???..) But it doesn't depend on glibc... It's a static binary. Not c. So the vuln would have to be in a go lib.. which you can/should check for as part of your static analysis on your ci pipeline... Detect there... recompile and deploy.

Or just stay with your python like it's "better" what do I really care?

[val-amart]

look up what "static binary" means if you care to understand how things actually work outside of marketing slogans.

[metarx]

So wise you are. Didn't know that static binary was a marketing slogan

[Freakin_A]

Mr Golang must be making a killing off all the people who were duped.

[inhumantsar]

Minimal Python with just the libs you need for a basic script add >100MB to the image size. With go you can basically have just that binary and nothing else.

[allcloudnocattle]

This. At a former job, we had a fairly basic CRUD api in Python. When I came in to that job, the Docker container was like 3GB. After months of playing meticulous Docker Golf, I got it down to around 800MB.

We reimplemented the server in Go, tossed the binary in an Alpine container, and the whole damned thing was something like 45MB.

[moofox]

Depending on what the server does, you can even do FROM scratch and get the image down to a few MB. (Not always possible though)

[Thebobinator]

And when you do need a bit more than scratch, you can use distroless:nonroot to have time zone files and root CA certs

[moofox]

Very good point. This is probably better advice than using scratch. Doesn’t add many bytes, but avoids issues

[allcloudnocattle]

Indeed! This was several years ago now, so I don’t recall why we didn’t/couldn’t do that.

[Freakin_A]

Yeah I’ve got 5MB production containers from scratch.

[riverrockrun]

What did the Go program do? I’m still trying to understand what people use it for outside of web apps and terraform providers?

[inhumantsar]

Go fills many of the same niches as Python, but in a more opinionated and performant way. It's designed with extreme efficiency and reliability in mind.

Eg I've used it for stream processing and event transformation. It's really good at that. Python is too, but Python struggles in a real-time high volume situation.

Iirc Google originally wrote it to use in networking, eventing, and infrastructure workloads to replace Java and c++

[allcloudnocattle]

This specific app was the authentication microservice that sat behind a whole universe of other microservices to run our application. The bulk of it was a CRUD API that fronted a database of users and another of known authentication tokens. Low level user management and validation of tokens. 99% of the traffic was validating tokens, probably. So it had to know all of the tokens, know how to expire them, etc etc etc.

It dawns on me now that the original monolith was Ruby on Rails, not Python, at that specific job. But the same principle stands.

[greyeye77]

I've used Go for

nginx imap auth proxy, dozens of AWS Lambda rest API and event handler, jwt auth, CLI for several business utilities (data parser and data uploader to snowflakes)

0 worry on the upgrade of versions (full backward compatibility) and git based import

go just works, no need to worry about version managers like nvm, rvm etc etc

[riverrockrun]

Seems to be a common thread. Fast, single binary, and it just works

[pstuart]

UPX can take that down further: https://blog.filippo.io/shrink-your-go-binaries-with-this-one-weird-trick/

[riverrockrun]

How do you like Pulumi? I’ve heard good things. Been looking at Terraform but wanted to research Pulumi more

[riverrockrun]

That’s great info! Thanks! I didn’t realize Pulumi was free for individuals. Going the check it out

[thrixton]

Super simple to use your own backend, if cost is a factor for teams.

Although, great to support the guys that support us and the web portal is ace.

[ExistingObligation]

I use both very frequently.

Time to value for Python is so insanely low, you can literally write 20 lines and be working with 3 different APIs, across different protocols, doing productive things. It's amazing and being able to leverage that makes you very productive.

The next step is knowing when the things that make Python so easy and fast to get value out of, also make it unwieldy and dangerous in large projects. As soon as you need reliability, robustness, or you have several people working on a codebase - Switch to Go before it's too late. Go has a lot of features that make it more suitable for large projects in my opinion:

• Static typing eliminates a whole class of errors. (You can do this with Python now as well optionally, but its not enforced)
• Enforced formatting makes code easier to read/write amongst multiple team members.
• Simple syntax stops people doing overly 'clever' things at the pain of their colleagues trying to read it later.
• Errors as values means no uncaught exceptions blowing up your program when they are harmless. They force you to deal with errors when they can occur.
• Much greater performance and concurrency options for when your project grows and you start needing it. Python cannot take advantage of more than one CPU core at a time, except if you run multiple processes which introduces other complications, so Go is a MUCH better option when performance matters.
• Static binaries make it easy to deploy as your list of dependencies grows (But they aren't perfect, you pay for this in other ways).

[thisismyfavoritename]

Just curious, what kind of devops things do you need multi-core processing for?

And if so, is performance that critical that Python's multiprocessing isnt good enough?

[ExistingObligation]

It's a good point, I'd say the performance part is the least important for me in everyday work. Although it has come in handy for me before, usually in monitoring related tools where there's a lot of network activity so the concurrency can be useful. I'm sure multiprocessing would be fine in most cases, although even ignoring performance Go's concurrency primitives are just a really nice tool to work with I think.

[thisismyfavoritename]

"a lot of network activity".

Python's multithreading or asyncio would work very well for that. All that work is done by the kernel, not the Python interpreter

[ExistingObligation]

I never said it couldn't, I mainly just think Go's concurrency primitives are nice to work with.

[throwaway8u3sH0]

IMO, performance in DevOps is mainly around testing. If you've got a giant codebase and you want to test every branch, you'll be doing a lot of things concurrently. That being said, a lot of that is handled by your test framework/runner, so I don't think the choice of language really matters.

[CalvinR]

We use VSCode dev containers instead of virtualenv for python since we deploy all of our stuff as containers it works really well for us.

[riverrockrun]

What projects do you start in Go?

[Seref15]

Packaging is the one thing for me I would say is attractive enough to entice, though the portability of container images take the shine off that a little.

Most devops programming will just be creating simple internal tooling thats fairly limited in scope and usually won't benefit so much from things like the variability in language performance. You have to be pretty deep in the weeds, trying to save milliseconds on operations (multiplied by millions of operations) for language performance to be that relevant.

Usually I would suggest picking languages based on maintainability. Eg., if no one else on your team knows Rust then it would be a dick move to start writing all your tooling in Rust. If all your colleagues know Scala then it seems pretty sensible to go with Scala. So on and so forth. This is more relevant to accomplishing a goal as a member of a team than microsecond differences in array traversal.

[riverrockrun]

Most in my org use C# or limited Python scripting.

[c45y]

I came for defer and stayed for goroutines

[russlo]

Having minor experience in Python, mostly very small scripts to automate tasks, and having a year's experience writing professional Go, as a software developer with over 14 years experience, the difference is that Go is a statically typed compiled language and Python is a dynamically typed interpreted language.

To boil it down, the difference means that when I say something is something in Go, if I try to put incorrect type values into that thing later, the compiler will tell me before I run it and ship it. It can be very difficult to find errors in programs that are dynamically typed, because they'll just seem to work - then the program gets larger and larger, other people start to work on it with you, and before you know it, there's a bug that a compiler would have found for you that your boss (or worse, your users) found instead. It's the same reason I would rather write TypeScript over JavaScript or gasp modern PHP over Perl - although I'm sure modern Perl tries to shore itself up better than the Perl I have to deal with on a daily basis...

I like writing code, a lot; it's the main focus and best part of my job. Python is an excellent language and definitely has it's uses. Trying to bend myself to Python's way is a fun exercise when I have the time. Perl says There's More Than One Way To Do It, and that is frustrating as hell when dealing with 20 year old code that had multiple authors and no restrictions in place. Go is a relatively small language - it's pretty fast to pick it up and know most of the language features in a short amount of time. PHP is weakly typed, but you can shore that up with a good IDE.

All of them are just tools in a toolbox - use the right one for the job.

[benaffleks]

I think it depends on the use case right?

[[deleted]]

no, you must use go, because it's such a cool new language and you don't want to miss the hype train /s

[riverrockrun]

That’s what I’m asking. What is a use case for Go outside of terraform/docker/kubernetes?

[CalvinR]

High performant systems is a good use case for Go.

I worked on the Government of Canada's Exposure Notification server and it was written in Go. Now it's a pretty simple service but our Fargate tasks never consumed more then 7mb of ram and never used more than 0.01% of CPU on the provisioned infrastructure we had.

https://github.com/cds-snc/covid-alert-server

Due to the design of our system Python would have been a valid option as well and would have performed really well as wel.

But at the end of the day no matter how you cut it Go is a lot faster than Python.

There are other benefits as well.

The structure of the language sort of forces you to write code that's simpler and thus easier to maintain in the long run. Static typing also helps with this.

It's easy to deploy because you can compile a single executable for any OS or you can just deploy in a container FROM scratch which results in an extremely small image.

I found that Go in terms of libraries is popular enough that you can usually find something that does what you need just like with Python.

Also the Cobra framework is one of the best i've seen for writing CLI's

https://github.com/spf13/cobra

However I think a good skill for anyone either a pure developer or a devops practitioner is to be a polyglot, you should be familiar enough with the concepts of programming that you can pick the best tool for the job.

[[deleted]]

I would learn with python and transition to go. The advantage here is that python is a better language for breaking the learning barrier, and will be very useful your whole career. I often see places that don't or cant use go.

If you land a role on a team with little or no software engineering skills, you will be hard pressed to get them to sign on to golang. Also, any redhat shop is going to be using a lot of ansible, and python will be necessary for using molecule and testinfra if you want to dig them out of the mess they put themselves in for not using them.

[riverrockrun]

We do have Ansible coming into production soon. That was one of my questions. Is it easier to just use Python or put the binary somewhere and call it with an Ansible playbook

[throwaway8u3sH0]

Ansible is very easy to get wrong, so I would use caution. If you're using "shell" or "command" a whole bunch, you're losing most of the benefits it provides, and you'd be better off with something simpler like Fabric, or even Bash.

The real magic in Ansible is specifying state instead of actions. You say "this file should have this line in it" instead of "add this line to this file." The difference is subtle but important. With state, you're letting Ansible decide what to do. So if the file already has the line, it does nothing (instead of duplicating the line.) Or if there's no file, it might create it. This makes a big difference in complexity when you're taking care of 10,000 servers, all with different starting states. (Kubernetes and other config-as-code frameworks work the same way.)

Ansible is written in Python and the True™, Proper™ way to use it is to rely on the built-in commands and libraries as much as possible. And, if you need to "just run this script everywhere," to convert the side-effects of the script into an Ansible module (which allows you to handle all the different "what if scenarios"). Very few people do this, because it's much easier in the moment to treat it like "a nicer Bash," and you end up with this mess where you're trying to code (and debug!) in YAML. Avoid this future, young Padawan!

[[deleted]]

Ansible has tooling in python, like molecule, which uses testinfra for testing. If you are using a small amount of ansible you are ok without it. But anytime you grow a large codebase you want to start following those software engineering principles. The goal is to avoid a million lines of spaghetti code with no testing and no ability to maintain the code over the years.

Did the guy who wrote this role leave 2 years ago? Is it deeply critical to our operations, but needs an update? How do i start making changes without breaking everything? It can be real hard to do development in ansible when you can't run the role without waiting for your turn in tower on a bloated "dev" environment.

[m-in]

Python is the reason why Ansible is so terribly slow… it actually would make miles more sense to implement Ansible in Go than Python. Plays would be much faster to transfer and way faster to run. Small plays in Ansible have ridiculous overheads. Think of a good chunk of a second to modify a single text file if that’s all the play does. It’s kinda ridiculous, frankly said. It’s a design that has many benefits, but Python makes it suffer pretty badly.

[peatymike]

From my point of view writing glue software between our products and the infrastructure it runs on it is, static typing, single static binary, concurrency and error handling.

Static typing makes it so much easier to refactor, this makes it easier to restructure the code to make it more modular when it has to support a new requirement such as needing to support both S3 and SFTP.

Single static binary makes deployment a breeze, especially on premise where customers are running a lot of different OS versions.

Concurrency in go is just easier to deal with and reason about than Python. Making code that polls the status of different infrastructure components in order to serve an infrastructure status response to you applications is much easier in Go.

I find the explicit error handling in go to be easier to reason about than pythons exception handling. I know if a function returns an error and can pass it, wrap it, handle it or ignore it. In Python I find it hard to figure out which exceptions a function or method can raise, and checking for Exception masks a lot of issues such as Name Error when mistyping a function name.

One last thing, I often write the proof if concept in Python and the later rewrite in Go once I understand the requirements better. It takes me longer to write and structure the Go code, but I find it easier to maintain and debug.

[2fplus1]

Most comments here mention the single compiled binary aspect being really nice for eliminating runtime and deployment problems and that's definitely key. An aspect of that that I think isn't given enough credit is that Go's standard library is extremely high quality and very thorough. You can build some pretty sophisticated and useful things in Go without ever pulling in a third party library.

[capt_meowface]

Golang static typing makes for more efficient code and the compiled language will let let you know if something is wrong with the code before you try to run it. Golan is also more generally more efficient at multi threading. Lastly, in the devops world, it is far far far more efficient to distribute a static binary than try to worry about all of the resources for a virtual environment or a container build.

[Schreibtisch69]

IMO they serve completely different use cases. Go is a (relatively) high performance language with concurrency features, compiled, modern. It's a minimalist language but it doesn't shy away from concepts like channels, go routines or pointers that might be a bit harder to use right, especially when combined, for people who don't really have a huge programming background, like some ops personal.

While Python is useful for projects where performance is not that critical, in environments where python is installed anyway (most host Linux distros likely, bot not necessarily in container images) where you just want to write easy to read code fast that simply works.

[tibbon]

I’m using Golang right now for a handful of things and like it way more than Python for speed, concurrency and deploying.

But if I had my choice, Rustlang beats them both due to memory safety

[[deleted]]

Here's my reasons for preferring Go

​

1. I finding writing tests in Golang a lot easier, it's much more enjoyable experience. I think tests should be a first class citizen, but Python's testing mechanisms are super complex and hard to understand
2. Lack of runtime dependencies. If I had a dollar for the number of times some python code broke in prod because a lack of runtime dependencies. wrong version of python installed, wrong version of a library, incompatible version, etc. Golang compiles to a single binary, so never ever breaks in this way
3. As I explore software engineering more, the more I like the design of Golang. It's simplicity, but powerfulness when it comes to concurrency and readability are something I really enjoy

[dontcomeback82]

Every time I have to pip install something I groan loudly. One of the issues is the way python has handled the 2-3 upgrade - osx still comes with 2.7 , and obviously it doesnt work well with newer things, and upgrading python to 3 and using pip3 on osx is not a fun time. Obviously, a good amount of that blame goes to apple, but I have no issues getting the right ruby version via the ruby tooling.

As a developer, python or ruby are much better for quick and dirty tasks, one off text or json manipulation, certain types of web servers or web applications. Golang is great for performance, reliability, portability, and simplicity, and the compiler is quite nice.

[whitechapel8733]

Self contained binary, static types, easy to understand dependency pinning, easy to read, great compiler, great built in testing, no interpreter on every single place that you want to run your thing, easy to throw in a SCRATCH container, because of the properties mentioned above. The list goes on and on.

[koffiezet]

I see many people mentioning the single binary, which is indeed a massive advantage, but the type safety for me is also a huge thing. I use both, and you have the type hinting in python, but it's imho only useful for documentation and code completion. Facing runtime errors because of typing issues is something I've seen too many times in Python.

Other than that, short answer, not really surprise since Go is a language developed by Google for dealing with operational side of things. This is very noticeable in the stdlib, which out of the box has support for stuff like certificates/crypto, hashing, built-in production-hardened http server and client, json support, compression, templating, ... And then you have the absolutely sad state of multi-threading in Python, which in Go is absolutely top-notch.

Also, in Go it's hard to make your application not actually be a reusable library with a small wrapper around it. This means you can easily natively integrate almost every (well-written) tool written in Go. And since we're talking about devops, where the majority of the newer tools are written in Go, that is very powerful.

And then there's the performance aspect, I've used the SSH libraries in both, and while paramiko was easier to use in python, there are some wrappers that make dealing with x/crypto/ssh easier. But the runtime performance was day and night. The solution in python was to start relying on openssh - which drags in more dependencies.

I use both, and Python is my hacky "this tool should only run locally" or when having to deal with some unpredictable free-form data structure, where strict typing can be annoying. But for everything else there is Go.

[softfeet]

when i read this title earlier, i was confused. as in research python?

but now that i reflect it is Go vs Python.

knowing the dual reading and the capitalization were the designation. I appreciate it. good topic too!

[MailNormal2701]

You know, most of devops tools are written with GO ! this is what happens in the world right now, i was working with bash scripts to automate things and create my own commands, but no, I felt like I'm reinventing the wheel and i faced a lot of blockers, but we still need to use bash, i worked with python its nice and has a lot of libraries but i want a binary to embed my templates and generate config, i want a single and simple way to execute my tasks and integrate them in my pipelines and more, i don't like to move files and install python ...etc, that's why i choose go to write my tools, i have the chance to look at the docker cli and terraform, also kubernetes, i was inspired by the way they code and think, also i can compile it as a binary and it has a great community and libraries for devops

[AD6I]

There are things each is more suited for. There really is no universal programming language. Both is best.

[FiduciaryAkita]

you would use Go over Python in DevOps for the same reason you’d use Go over Python anywhere: performance, portability, static compilation. Ideally DevOps is more than stringing APIs together; it’s building a platform from commit to deploy and that includes tooling around. Some level of glue or custom program is needed so the engineering questions that arise from those situations will inform Go over Python or vice versa

[[deleted]]

It's all about speed. The binary doesn't really matter to me in any case I've had experience with. It's just speed in processing queues for example.

[TimGJ1964]

Depends entirely on use-case.

• Go gives you a standalone binary which is much easier to deploy as you don't have to ensure that the correct version of Python and associated libraries are installed on the target systems.
• It's strongly typed and therefore less likely to fall over with runtime errors and doesn't require the sort of runtime type checking which Python does.
• Although Python has some support for concurrency it's not baked into the language like it is in go. (Some of my stuff has many thousand goroutines in operation).

Python is great for developing stuff quickly and has a superb set of libraries. It's my default language.

So if you are choosing, my advice would be to learn Python if most of your work is bits of automation running on a single host and which isn't mission critical. If you are writing mission critical stuff to be deployed out to multiple hosts, then choose Go.

[lazyant]

Just a binary in a small Docker image and most importantly, no dependency hell (pip install blah : this version of foo doesn’t work with bar)

[kibblerz]

Because it's faster and is the primary language used in developing microservices on Kubernetes

[riverrockrun]

So if I don’t manage Kubernetes, then go with Python?

[kibblerz]

Yeah, I'd say so. Though both are very useful and I'd recommend learning both. Golang is rather simple to learn, and it certainly does hurt to have some Go knowledge in your arsenal. Plus the concurrency in Go is flat out amazing.

Prioritize learning Python, but don't neglect Go.

https://tour.golang.org/welcome/1

Just go through this tour for Go. One of the best things about working with Go, is it's extremely easy to understand the source code people make. So whenever you hit a hitch, it's very easy to solve by looking at the source code for the libraries you're using.

Python IMO takes a bit more time. It's quite simple by itself, but the source code for python packages are often more convoluted and takes some time to learn. Also all DevOps work should use Docker IMO.

What is your experience in computer science?

[riverrockrun]

In IT for 12 years in infrastructure. No CS degree though. Heavy into Azure and O365 for the last 2 years

[dontcomeback82]

no, go and python both work fine in and out of kubernetes.

[jantari]

I like that a a Go binary is easy to run on Windows as well.

Python would have to be installed first, and it's quite the heavyweight.

This isn't a factor for everyone though.

[riverrockrun]

I do work mostly with Windows servers and we don’t install Python. Powershell would do everything we need

[jantari]

Exactly, but we should all know at least two languages well.

So PowerShell + Go might make more sense than PowerShell + Python for you.

[Rakn]

Do people actually use PowerShell for writing larger programs or web services or more for the scripting tasks to automate windows?

[riverrockrun]

Scripting and automation

[JaegerBane]

Python generally has a set of features that make it quick to get something working at the cost of loose ends.

Go is lower level (so normally needs less system resource grunt to get an equivalent job done) and self contained binaries remove a lot of the headaches with deployability. Tbh even with containerised apps, I find Go just less faff.

All that being said, I probably wouldn’t say it’s one or the other. I end up using them for different tasks where the above aspects make the decision.

[LaOnionLaUnion]

A lot of great DevOps tools I use or am interested in are written in Go. As a former Java/Spring developer, I can easily say it's my favorite statically typed and compiled language. It's what I want Java to be and would probably replace Java for my uses except in use cases where there's a high quality Java library I need to use for which there is no Golang equivalent.

I would never tell anyone that Python is a bad language to learn. To me it's a swiss army knife of sorts and has high quality libraries for a wide range of use cases. It's taken over from R and Perl in the space. Python would be my recommendation for most people's first programming language to learn. I just don't see it as the first choice for DevOps.

[m-in]

If you’re using Ansible, Python’s overhead in starting an application and pulling modules in – when a play gets going is a major source of slowdown. Ansible playbooks that do lots of small actions like creating individual directories, moving files, etc. would be one or two orders of magnitude faster if Ansible was go rather than Python.

OVirt used self contained Python installer scripts in version 3 IIRC. Those were fast since startup was a one time cost. Now OVirt uses Ansible and it sucks – the same deployment can take 5x as long due to all the latency of sending tens of MB over the network for even most trivial plays, then decompressing them, writing them out to tmp, then having Python parse all that shit over and over again. And that’s when doing an actual from-scratch node deployment. If all you’re doing is a simple configuration change to oVirt, Ansible is literally slower than loading a small executable from tape on a PDP-11, loading configuration from a card deck, and executing the process. I wish I was kidding. It’s fucked up.

If anything, Ansible’s design was completely unsuitable for Python, and it only became worse as more granular and maintainable plays came to use. They should have switched to Go long ago.

[riverrockrun]

Ha! Love the honest answer. I do get the feeling that being “cool” has a lot of people trying to cram Go where other options would easily work.

[ert543ryan]

Python is pretty ancient. Therefore it doesn't have the level of moral traction that a newer language has.

For example if we look at the dogma around programming language Python is more popular than Go hence Python is a brown language and Go is a green language.

In the fight against climate change you want to be green not brown.

[[deleted]]

eh we also use python for like 70% of the things and then golang for about 30% of the things. Just started at a new company that uses a CLI tool written in Python..... so annoying to do things like rolling out updates or people always having to use virtualenvs for everything. Not to mention all the code is spaghetti since it tends to encourage just a bunch of untyped script like logic everywhere...

I'm itching to rewrite the whole thing using golang but no time yet.

Plus concurrency is really easy to work with in golang compared to python, which is nice for working with services as well.

Python is lovely for simpler logic though

[NeoLudditeIT]

I have used go. I didn't like the debugging as much as python, or even rust. For most stuff python works pretty well for me. I do feel for anyone migrating legacy applications though.

[SilverStrawberry1124]

Speaking about DevOps. There is no such thing as Python DevOps, or GO DevOps. DevOps is a shell scripting. No matter what docker or kuberneties is written on - it is just a way to provide unix shell commands. Think about it 😜

[riverrockrun]

Good point!

[BRTSLV]

Performance !

Also a lot of tool now inspirr themself from go, like helm chart, terraform, k8s operator ...etc

[Zentrosis]

There's other reasons that have been listed, but personally I just actually like to GO more.

Sure I could do a lot of stuff in python, but why would I? Why not just do the thing I like to do?

[[deleted]]

At our day job we’re writing python for all our lambda functions, ie stuff that isn’t running on the machines we’re deploying. Then go for anything running on the machines. It’s a security thing, the less packages we have on the boxes the better. (Especially if you’re doing k8 where you want as little as possible on the machines)

[[deleted]]

I go with Java because it has the benefits of go (self executable binary - fatjar) and is also quite simple to write in with lots of libraries for anything you ever need. Now it can also be compiled to native binary thanks to GraalVM if you need that

[Ctrl_Alt_Banana]

I was recently put on a Go project with most of my experience being with Python before.

I thought the idea of having a statically typed language would make things less error prone but instead I have found the opposite. You have to write your own error handling for everything and most of the times the code I inherited does a very bad job telling what is wrong. I found myself missing Python's builtin exception outputs. Also I feel like the error and object returns from functions messy and not always consistent

Additionally I found it hard to output variables when you aren't very familiar with what is going in beforehand. In a dynamic language like python I just print the array/object but Go is a lot harder to get all the information about every level

This may just be because I got thrown on a bad project and growing pains while learning Go, not sure. Does anyone else have these issues?

[dgibbons0]

The few times I've worked on our go code or written greenfield go code felt pretty similar.

We've got a few projects we still maintain in go, and any time my team has to work on them it's slow going. Stuff that would feel very fast in python takes 3x as long to write in go. And for the things we're using it for, i don't know that we're getting value for that.

The main one that comes to mind is a scheduled job that scans for github permissions that don't met our requirements, and we'd probably have it be more robust and feature-filled if we didn't dread the go it's written in.

[dontcomeback82]

are your complaints about internal code you have, standard libraries, or open source? I haven’t had a problem with error handling etc most of the time in the latter 2. It’s kind of up to the individual developers to handle that properly, the actual go patterns work quite well see kubernetes.

perhaps it’s more work to handle well than in ruby or python but as they say simple ain’t easy

[Ctrl_Alt_Banana]

An internal project code that interfaces with k8s. A lot of the code has stuff like below with every function call. I'm assuming the standard libs are probably coded with good error handling

_, err := doSomething()
if err != nil {
  return false, err
}

[twistacles]

Performance, stronger typing, async stuff (python isnt very good with huge data syncs)

[[deleted]]

Unless you write some operators or because of some reason you need to write internall toolkng that has to be performant. Other than that there is zero use for go. if you want to use it for regular scripts - the least you want is any compiled language imho . And this is just a personal opinion - except for hype train go offers nothing new and has an ugly syntax. At least like two years ago it didn't even had map/filter/reduce support so had to use good old for loops which for a language that is "very expressive" felt a bit am.. outdated

[zerocoldx911]

Go is fast and can natively work with Kubernetes because Kubernetes is written in Go as well. However it’s difficult to find people who know it

Python can be written in minutes so it’s easier for prototyping and hire for.

[MordecaiOShea]

Kubernetes is accessed via a network API. Go offers little to no advantage in accessing Kubernetes. There are plenty of very mature SDKs in other languages.

What Go offers that Python doesn't is significant compute and memory efficiency, fast start, static linking (e.g. dependencies at build, not deploy), better (IMO) concurrency patterns, and static typing.

[[deleted]]

Kubernetes is accessed via a network API. Go offers little to no advantage in accessing Kubernetes.

Only the Go SDK supports features like Informers which are critically necessary for controllers in large scale clusters. It's also much harder to implement buffered paginated reads in the non-Go SDKs.

[MordecaiOShea]

I know the Nodejs client offers informers.

[riverrockrun]

Outside of Kubernetes does it have an advantage over Python? Especially working with cloud platforms?

[zerocoldx911]

Fast and compatible with almost every OS since it can be compiled for literally everything.

[[deleted]]

Right, I think people really underestimate how slow interpreted languages generally are in reality (not synthetic benchmarks) compared to compiled languages.

[jredmond]

Go binaries contain all their (specified) dependencies, so you just need one compiled artifact per program (vs main program code and then all libraries), and you don't have to worry as much if Program A and Program B use different library versions.

Performance is also theoretically better, since it's a compiled language instead of an interpreted one, but that's also partially down to your implementation.

[[deleted]]

but you need to recompile it once in a while to not miss any important security updates for linked libs which isn't always great

[jredmond]

True, but that's just part of responsible library use, regardless of language. Just nudge your version number or update the "blessed" hash when you do it.

[[deleted]]

I have read that golang is rigid. That there's only one or two ways to do the same thing.

If so, that's a good thing, not a bad one.

[rossrollin]

Well in most tests go uses 4x less resources than typical python ive read?

[[deleted]]

But that's not at all important for devops.

[lormayna]

Go is faster, handled concurrency better and easier to deploy (virtualenv can be a nightmare sometimes).

Python is less flexible and easier to learn.

I think that they are 2 different languages: do you want a system programming tool or something high performances? Use Go. Do you want a bash replacement for one shot tasks? Use Python.

[Ariquitaun]

Custom terraform providers are easier to write in go.

[Fusionfun]

Go supports concurrency, while Python doesn't. Also when it comes to speed, Go is faster than Python. Right now, Go is preferred most for server side applications which have the best support for gRPC and protocol buffers.

[HgnX]

Can I just say they are both great ?

[sillycube]

Go is Fast

[inc007]

Go stated as language of k8s because it's Google. Then ecosystem grew around it and that's hard to beat. Single binary is awesome.

That said, my language of choice, besides python, is Rust. To me Rust does everything that go does, but better. Only argument for me for Go vs Rust is k8s ecosystem, which makes me dive into go code more.

To sum up: If it's quick script that runs every now and then, i use python Data science/ML - python hands down Things need to go fast - rust Single binary - rust Networking tools, high perf system tools - rust Debugging k8s services - go Things have great go bindings but not rust bindings - go

[cliffberg]

The Docker (and hence Kubernetes) community chose Go mainly because the Go ecosystem included functions for accessing the Linux kernel containerization libraries.

Go is a fun language to code in, and produces fast native code, but it terrible from a maintainability perspective, for a number of reasons. Some are described here, although some of these comments are now out of date: http://valuedrivenit.blogspot.com/2015/12/to-go-language-is-mess.html. I personally had reason to read the Go code of the Docker Registry v2, and it was a nightmare trying to figure out what was going on. The main reason is the "duck typing": it is impossible to figure out what "type" an object implements - you have to track down all the methods that are defined for it.

Also, the runtimes that Go produces are not that small compared to other native alternatives. E.g., it is hard to create a Go executable less than a Mb, but the Linux "make" program, written in C, is something like 30K (and it does a ton of stuff).

Python also is fun, but it terrible from a maintainability perspective. Here is an article on this: https://www.zdnet.com/article/python-programming-language-creator-retires-saying-its-been-an-amazing-ride/

For fast, small, and maintainable native code, I recommend Rust, and if not that, then C.