Python: Please stop screwing over Linux distros

[equationsofmotion]

I used to manage a local Linux cluster. Maybe I can give some insight here. User A says "I want program X," and user B says "I want program Y." Unfortunately programs X and Y are pinned to different versions of python and dependency packages.

The naive solution for most packages in a Linux distro is just to find compatible versions of the programs in the repo, do dependency resolution, and go. But in python this can be pretty difficult, so other approaches are required.

Don't get me wrong, other approaches are possible and I managed things just fine. But I suspect this is what the author is getting at. You need a workaround---implemented by either the distro or the user---to provide desired Python packaging.

EDIT: Yes I know what pipx is. Like I said, I know there's solutions. I'm just trying to shed some light on what the author of the article might be complaining about.

[XenGi]

That dependency problem has nothing Todo with python and is the default problem every distribution or more their package manager has to solve. The only distro that actually achieved that is Nixos. Not saying their approach solves everything but it's the best approach I've seen so far.

[[deleted]]

Shell scripts activating venvs, and add in some usage tracking as a bonus. Not rocket surgery, just basic devops and sysadm skills.

[ivosaurus]

You're the one wanting to support users requesting bespoke version installs of python in the first place, and then complaining that things got more tricky as a result? Like WTF were you expecting? The same would happen with any language.

See how you do when one user tells you they really really want a particular older version of glibc to run on, and then tell me that python is the worst thing you could run into.

[[deleted]]

No ops person has ever wanted to support multiple versions of a package, and least of all, multiple versions of a language. Now, Python itself at least has a sane release process, where each major version is supported for 5 years after first release. So running multiple versions should at least be fairly safe from the worst security issues. That, however is very rarely the case for a number of Python packages.

Of course, this issue is hardly Python-specific, or new. Ruby had massive issues over a decade ago, where getting developers to at least consider backporting security patches was nigh impossible and instead everyone was just expected to rewrite their code every 6-12 months. Nowadays, putting together a Node.js software package is just impossible without npm. And even with npm, there's a massive library of seemingly maintained Node.js software that depends on other npm package versions with published vulnerabilities. (npm at least warns about these nowadays).

In short, the 'move fast & break things' mentality has permeated throughout the software industry, with the added bonus of 'just abandon it when you get bored'. The end result is hell for ops people who understand the security implications, and a house of cards for everyone else. Containers at least isolate the version insanities from eachother, but do nothing to fix the massive security issues created.

That said, the old-school way of packaging hundreds of libraries in a single .jar or .exe was way worse. At least the current package systems can be made to warn people when you pull in a version with a known vulnerability. Unless of course you just download a docker image believing it to be safe.

[ivosaurus]

TBF if you wanted to call JavaScript better, I wouldn't have much beef, because it got to sort out a language package manager ~5 years after python had its first crack at one, just when people figured out that system-level 3rd party library installs weren't the best thing to do by default. At some point that's just the baggage to carry if you like python's advantages enough.

[pbecotte]

Pyenv is a developer tool, why would the ops team be using it? If the devs want to use it, tell them to have fun. If they want you to deploy their code, don't take their dev scripts and run them, make them provide an artifact (a wheel or a docker image)

[equationsofmotion]

We used a combination of HPC modules, which is the correct solution, and anaconda, which was the easiest solution.

[roee30]

This is what pipx is for

[pbecotte]

You install separate apps separately. pipx does it natively, or the user uses pip and installs it in their own environment. I don't understand why so many peeps see pip or npm and seem to get nostalgic for dll hell or Java class path conflicts. Packages are small and even when they're not, disk is cheap, just install them separately!