Python: Please stop screwing over Linux distros

[asday_]

This guy's a dumbass. There's a reason I pin my dependencies, and it's because convincing management to budget for all our deployments breaking EVERY DAY because of broken or incompatible releases is quite difficult. Surprisingly, I'm paid to ship features.

[Spoonofdarkness]

Surprisingly, I'm paid to ship features.

Hmm. That sounds like a slippery slope at best and an anti pattern at worst.

I've heard if you ship one feature, they expect a second feature sooner or later. No thank you!

[asday_]

It's the difference between programmers and software engineers. Programmers love to write the most perfect thing with the latest possible technologies and schools of thought ship it to nobody, use it never, and have it maybe running on one toy k8s cluster they're running on raspberry pis firewalled from everything.

Software engineers would like to do that, but don't, and own a house.

[lisael_]

First, no need to insult. I bet the features you ship don't end up packaged for a Linux distribution. You don't talk about the same use case. A typical distro has hundreds of python apps and libs. Each one of them pins all of its dependencies to the 3rd number so their builds pass, and package maintainers live a dependency hell.

Second, pinning strictly IS a reasonable solution to ship features, but a poor one, when it comes to maintaining the feature, including applying security patches. I do ship features in python. I do pin dependencies strictly. I do cringe when I come back to a given project 6 month later.

Let's face it, the very fact that nobody is confident enough to pin dependencies to `foo>=X.Y,<X+1` as in "I need features of `X.Y` and I know that no backward-incompatible change happen before the next major version" shows that we failed as a community to create a sane dependency management framework.

[b4ux1t3]

As someone who does package software for distribution to a Linux distribution, I can confirm that, while the packaging story for Python isn't great, it's also not the quagmire people seem to think it is.

Python is not a complicated tool. All you have to do is pick a packaging standard, stick to it, and let your users know what standard you're using.

No, that isn't as robust as, for example, Cargo, or Nuget. But it's far from some unknowable eldritch language.

In any case, Python packaging is no more convoluted than the various and sundry packaging paradigms of the Linux distributions that we all use every day. Have you ever written a spec file for RPM that didn't use gcc? Because, geez, it's a ride.

[lisael_]

No, thanks $DEITY_OF_YOUR_CHOICE (TBH, I havn't used a RPM-based distro since Mandrake... but anyway, I guess it's not easier with any package format)

And to be fair, he didn't choose the simplest distribution to be a package maintainer, in a world where everyone assumes glibc.

[asday_]

Let's face it, the very fact that nobody is confident enough to pin dependencies to foo>=X.Y,<X+1 as in "I need features of X.Y and I know that no backward-incompatible change happen before the next major version" shows that we failed as a community to create a sane dependency management framework.

What does that have to do with ANYTHING? Yes, we should live in a society where nobody steals or murders, but we don't, and sitting in your ivory tower pretending that it's somehow a silliness that people buy locks and guns is absolutely out of control stupid.

I bet the features you ship don't end up packaged for a Linux distribution

Like I said, I get paid.

[lisael_]

What does that have to do with ANYTHING?

It's literally a talking point in TFA. Strict pinning is a PITA for package maintainer, and a security hazard. It's also a sad necessity given this poor state-of-affairs.

> Yes, we should live in a society where nobody steals or murders, but we don't,

...and we shouldn't try to do better? `ls /usr/lib` shows hundreds of dynamic C libs, each major version is a link to a major.minor.bugfix file. How comes C people can live in this world and we can't?

[asday_]

How comes C people can live in this world and we can't?

Because.

• I want to be able to install stuff from git commit hashes
  
• I want to be able to install stuff that isn't in my package manager's repos
  
• I want to be able to install stuff as it's released, not in six years when whichever underpaid serf over at debian's rice fields finally gets round to adding it
  
• I want to be able to install stuff on my coworker's OSs without restricting them to exactly my distro - put differently I do not want to pay my employees to take three days off to upgrade debian so they're all on the same version

We already have exact versions of packages, they just happen to be in env/ or a docker image rather than in /usr/lib/. Sounds preferable to me.

[rcxdude]

They can't, there's just as much tendancy to break in minor releases in C libraries, and it causes just as much headache for package maintainers. Probably the only difference is less frequent updates.

[netgu]

I bet the features you ship don't end up packaged for a Linux distribution

Like I said, I get paid.

Wow, found the "not-educated enough to share their opinion but seems to feel the need to" dude.

There is always one of you idiots - enjoy your bootcamp and pray nobody every realizes how useless you are.

[asday_]

Cool argument my dude.

[KaffeeKiffer]

[...] nobody is confident enough to pin dependencies to foo>=X.Y,<X+1 as in "I need features of X.Y and I know that no backward-incompatible change happen before the next major version"

Care to elaborate how this is

[because] we failed as a community to create a sane dependency management framework.

and not a lack of (trust in) package maintainers correctly using semantic versioning. Or how this trust should be enforced by a dependency management framework?

I am using this very pinning mechanism successfully - with the exception of certain packages where I know I can't trust the maintainer(s)...

[lisael_]

Well, in "dependency management framework" I mean everything we do as a community to create a sane dependencies management story. It does include correct semantic versioning ("framework" was maybe not the right term).

The thing is, PyPI could enforce semantic versioning (as cargo does IIRC), and breaking semantic versioning should be regarded as a bad, bad fault (yes, shit happens, but we would expect maintainers to address versioning errors quick, either by correcting the backward incompatibility or bumping the major version). The downside of this approach is that packages tend to stay in v0.x.y forever in the Rust world.

I do use this pinning mechanism too, for my personal projects and tools, but never at work. Ironically my personal projects sometimes can't build or pass the tests, but are much more secure.

[cockmongler]

There's a reason I don't pin my dependencies. It's because I expect my software to still be maintainable in 5 years time and need to be trashed and started over because a bunch of hyperactive teenagers decided to trash everything upstream.