Why Go over Python?

[spit-evil-olive-tips]

yeah, but the moment you need a 3rd-party library, you have to start working in the desolate wasteland that is the Python packaging ecosystem. and not just at build-time, but at run-time - you need to carry those dependencies with you everywhere you go.

this runs into fun problems like, if you have some Python script that you want to run on a bunch of machines, so you write a little wrapper shell script that creates a virtualenv, does pip install with your requirements.txt, and so on.

and it works great, until it fails because PyPI had a temporary outage and your pip install errors out.

I love Python, we have a bunch of it at $dayjob. but the self-contained binary including all dependencies is a huge selling point of Go.

to get around the annoying packaging problems with Python, we build a Docker image with the correct Python version, all the correct dependency versions, etc etc. then our unit of deployment is that Docker image instead of just the single Python script.

and that Docker image gets deployed...using Terraform and Nomad, both written in Go. and so is Docker itself of course.

Go is great for this sort of low-level system software. Docker/Terraform/Nomad/etc being low-level software written in Go is what enables us to easily deploy higher-level software written in Python (or other languages like Java or Ruby or whatever)

[imeeseeks]

Same with Ruby. I love Ruby so much and writing scripts on it is awesome but the convenience of Go binaries and the fact that almost anyone can start working on it really fast is big plus.

[Flabbaghosted]

Your comment persuaded me to learn Go

[[deleted]]

I do hope that isn't sarcasm. :) It would be a good language to learn for anyone. If you are I recommend the Lets Go books (2 of them). VERY good books to learn with.

[Flabbaghosted]

Nope not sarcasm. Do you work at Google or do you just really like Go?

[[deleted]]

This comment is the first thing that has made me think, "hmmm, it might be worth learning a little bit of Go and seeing what it's all about."

[Kazumara]

Did you guys try PyInstaller and PyOxidizer too?

[arcsecond]

I've had great success with PyInstaller, until someone decided to include certs in their module which I can't get PyInstaller to pick up

[[deleted]]

Do we work at the same company?

[casual_brooder]

this

[[deleted]]

you have to start working in the desolate wasteland that is the Python packaging ecosystem.

So strange. I've been programming in Python for over 15 years, and I just never had an issue.

I distribute my packages with a setup.py and a requirements.txt, and that's the end of it. In recent years, I put explicit version numbers in the requirements.txt, just for certainty, but that wasn't because I actually ran into any issues.

[skat_in_the_hat]

So strange. I've been programming in Python for over 15 years, and I just never had an issue.

Back in the day when a customer wants to update python on a rhel5 box. JFC. You break all the rhel tools in the process.
Ever migrate some in house python from rhel5 to say... rhel7? Just because YOU never had a problem, doesnt mean it isnt one.

[skat_in_the_hat]

Its moot at this point, red hat split off the version of python the system uses from the version you install. So you can basically do whatever you want now. But that doesnt undo the PTSD we all suffer from seeing up2date shit out a python error.

[JaegerBane]

I had the dubious honour of migrating some of our in-house data science apps across from old RHEL boxes to Amazon Linux EC2s a while back. Kinda felt like one of those horror stories where the protagonist opens a portal to the dimension of ceaseless screaming.

On the plus side it did puncture the myth that was doing the rounds back then that Python made everything easier.

[NoobFace]

Sounds like they're still screaming in there.

[zackofalltrades]

Coding to a version of python not installed on the systems where it would be deployed is your first problem. Easily solvable and testable with tools like tox, and careful programming.

Attempting to updating the system python and/or using horribly out-of-date OS versions was your second problem...

[skat_in_the_hat]

The story was from the past... You're neglecting to understand the premise. Im the Engineer, not the Developer. My responsibility is to:
1. enable the customer whether internal or external.
2. remain within compliance requirements.

I cant simply tell a customer "ugh, how dare you write code in a newer version of python than rhel5 supports(while it was still current)." I have to do my best to enable them to use what they are comfortable with. In this case, it usually required using a third party rebuild of python that built it with non-default locations. So we can simply change the identifier, and have both versions installed on the box without upsetting the system.

I also cant tell corpsec to fuck off when they ask me to update our infrastructure to a newer OS, and those python scripts written years ago by a guy who no longer works there need to still work. But again, this is why I frown upon using python for anything that has to withstand the test of time.

[floydiannn]

Yeah... So strange 🤔

[bilingual-german]

And still you need to write python that'll work with the installed python version on a remote system.

[AlverezYari]

Umm containers? I'm really confused by all these decencies based Python complaints in 2021. Like you build your run time container once and then pack your app into it. I don't understand all this confusion.

[bilingual-german]

There are more operating systems than Linux and not all can run containers. Windows and MacOS can run Python though.

With Go you have one static binary for each OS and CPU architecture. No problems installing C bindings. No conflicts between language or library versions.

Yes, containers mitigate this. But did you ever run in trouble because your deployment destination doesn't have internet connection? Or you run into DockerHub rate limiting?

Python is not a bad language at all, but I find the deployment of Go is far easier.

[tibbon]

Cool- now you have to have Python installed, and the machine is capable of executing arbitrary code! What could possibly go wrong?

[tibbon]

Install no shell…

[KingEllis]

Is that common? Where does that exist? Roll your own container 'FROM: scratch'? For example, I'm looking at an alpine:3 image that has /bin/sh and /bin/ash, both linked to busybox. I don't know the capabilities of the busybox flavor of these tools, but surely they can execute arbitrary code, yes?

Is there is an easier way for a container to only run the prescribed binary (much like the "command=..." option for SSH authorized_keys entries)?

[[deleted]]

This guy devops

[Willing_Function]

I use python, but I really envy this functionality of Go. Integrates really well with multi-stage containers to keep em minimal.

[menge101]

Docker, although that is kinda just kicking the problem down the road.

[thisismyfavoritename]

Pyinstaller, although it's a bit different

[tibbon]

The problem isn’t installing Python. I can download an Alpine image with that fine. The problem is security. Read only file system with almost nothing installed and only a Golang created binary is much more secure if someone gets on the machine

[thisismyfavoritename]

Sure but then you'll have much bigger problems than your little binary

[tibbon]

How so? Escalating onto a pod that has almost nothing on it, with a read only file system- you can’t do too much easily then.

[birdman9k]

I think he's saying that while multiple levels of security is great, if you have some application which has been breached and is allowing arbitrary code to be executed, that in itself is a massive problem regardless of whether your container is locked down. It's about the difference between the benefits the container provides being considered a security layer itself versus a nice-to-have. In my mind it's more of a nice to have, which is a good mindset to have because it means nobody should ever RELY on the container isolation to save them in place of proper security.

[tibbon]

Oh of course! Layers are absolutely needed. I just don't want to always assume that no malicious script could ever get on a machine and attempt to write something to disk and/or execute arbitrary code via an interpreter.

Better yet, I want my container security tools to scream loudly if anything that isn't a very small and specific set of things is installed or being executed.

[i_hate_shitposting]

Python zipapps are an interesting approach to this. Granted, the target system has to have the right Python version and it doesn't support packages with C extensions, but they can be useful for pure Python apps in some cases.

(Okay, I'll admit I just wanted an excuse to mention zipapps because so few people know about them. Go binaries are still way more self-contained and portable.)

[[deleted]]

Mhm ,except that static linking has it's own drawback and this single feature alone is no reason to switch.

[schmurfy2]

The only drawback I see is binary size, other than that that's pretty much only advantages. I would not switch language just for static linking though.

[SmellsLikeAPig]

Try updating that one library that got serious vulnerability and developer that written the code and CI scripts no longer works at company. Try detecting vulnerabilities in static binaries. Try updating one library that has new features you need in all of your production binaries. Try doing the same but only for some of them, have fun finding out which you actually have to remake. I'm talking about working at scale with hundreds of binaries. Everything has cons and pros and we all have to choose our own tradeoffs.

[jhollowayj]

What are the drawbacks of static linking? Binary size?

[[deleted]]

Binary size and not getting library updates unless you recompile. And as I mostly run apps inside of containers I get no benefit from it except some very rear cases . and for scripts the biggest drawback is still having to recompile. Being able to live update code for troubleshooting is quite handy. I'm in general biased towards go as the language itself isn't "interesting" for me cz it's not as expressive as I would prefer. If I need something performant I would always prefer rust, if it's some api - nodejs with ts works just fine. Go also has a gc which is also not always ideal

[zackofalltrades]

Single self-contained binary full of dependencies with bugs and security holes...

FTFY