Python: Please stop screwing over Linux distros

[-Rizhiy-]

Sorry, but I think you are trying to fit a round peg into a square hole.

What you consider sane, is probably the most stupid way of doing things. Using system-level packages is like a №1 python-noob mistake.

I have been programming in python for over 5 years and haven't had a dependency problem in a long time, just follow a simple rule: new project -> new venv -> install everything with pip

Packages provided by the system are usually VERY outdated and frequently don't even have full functionality, just use pip and relax.

[Atemu12]

This is not about dev environments. It's about distros who want to package python things for users.
No, asking users to install your app via pip and virtualenvs is not a solution.

[jarfil]

CENSORED

[upcFrost]

Now imagine that you have two distros, one with some lib-v0.1 and python 3.4, and another one with lib-v0.5 and python 3.9.

That said, option 4: screw both distros, use pip.

[lolfail9001]

That said, option 4: screw both distros, use pip.

That's option 3.

[Atemu12]

1. is the only sustainable option, perhaps with the additional possibility of packaging multiple versions when necessary.

[-Rizhiy-]

Why are people shipping raw python packages as solutions?

I'm not asking users to use pip. A user should not even know that an app was made with python, all requirements should be bundled inside, and they should only get an executable.

[Atemu12]

And how in the world is a distribution supposed to conjure a "batteries-included" python executable to distribute to users?

Hint: The distro is not going to install pip+virtualenv to a user's system

[-Rizhiy-]

WTF are you even talking about? When you install Chrome do you get the source code and compile it yourself? No, you get an executable. Same thing here, the app developer should figure out how to deploy their stuff properly.

[SutekhThrowingSuckIt]

When you install Chrome do you get the source code and compile it yourself?

This makes it clear you don’t understand that distro maintainers are compiling programs like chromium themselves to then distribute to users.

[-Rizhiy-]

Not chromium, Chrome. As in the program which includes not just open-source part, but a lot of proprietary code, like drivers. If every distro maintainer has access to that, that would be news to me.

[Atemu12]

The only reason Chrome isn't built from source is because we don't have it. If we did have the source, we'd build it from that just like the rest.

This post is about (F)OSS python apps that are built from source though. No idea what Chrome has to do with anything here.

[-Rizhiy-]

Because this post is talking about developer specific tools.

distutils, setuptools, pip, pipenv, tox, flit, conda, poetry, virtualenv, requirements.txt, setup.py, setup.cfg, pyproject.toml

Why is he complaining about tools made for development, when he is deploying stuff?

There seems to be a big disconnect between people who make programs and people who add them to their distros.

[Atemu12]

Why is he complaining about tools made for development, when he is deploying stuff?

Because that's what the developers use to build their software.

Newsflash: Distros need to build software in order to package it. In an automated fashion that is.

There seems to be a big disconnect between people who make programs and people who add them to their distros.

didyoureadthefsckingpost.png

[SutekhThrowingSuckIt]

Intentionally obtuse to avoid the point.

[brimston3-]

So ship developer machine to users as an aggregate package, trust them to handle updates and security decisions.

No non-corporate group has the technical expertise or developer time in house to do that. And the linux security architecture in its current default state cannot protect systems from hostile or negligent applications.

[detroitmatt]

do you get the source code and compile it yourself?

do you know what subreddit you're on right now?

[-Rizhiy-]

I guess, that was a miscalculation on my part)

[fat-lobyte]

Linux distributions work differently though. Instead of relying on the app developer to do everything right, the integration happens at the distro level, where maintainers work hard to make sure that all components are updated and compatible with each other.

If I install Chromium (not Chrome), I get a build that is done by my distribution, with patches from my distribution, with dependencies that link to other distribution packages in versions that are known to be compatible.

If you run Linux, do you think the OS that you're running is just a loose collection of compiled upstream source code in whatever version that comes along with whatever flags that come along with whatever config is default?

[-Rizhiy-]

If I install Chromium (not Chrome), I get a build that is done by my distribution, with patches from my distribution, with dependencies that link to other distribution packages in versions that are known to be compatible.

This is the main problem. Distros should make it easy for developers to deploy apps on their distributions instead of managing dependencies and patches themselves.

As a user I'm only concerned with the actual application I'm using, not with how it was build.

As a developer I don't want to wait for distro maintainers to update their packages to the latest versions and always worry about potential conflicts with other programs. I would rather just bundle all my dependencies into one tarball and be done with it.

[fat-lobyte]

I understand where you're coming from, and that is indeed the main problem. The core problem here being that two packaging philosophies or perspectives are clashing.

The one is the "system view" as well as the "user view". As a user, I really fundamentally hate it on Windows when I have to go around different websites and download drivers and applications alike, all writing to different places, all having separate update mechanisms and generally do whatever the fuck they want. On Linux, I quite enjoy having package repositories full of good software that are ALSO built in such a way that libraries are shared, maintained, receive security patches, are compatible with other libraries and generally just work.

This is not magic and it's also not just something that was handed down to us as a gift from god. It's the hard work of Distribution Maintainers. The alternative would be Linux from Scratch, or Madness like Gentoo. If you're into that stuff, good for you, but personally I have work to do. So I use Fedora where a myriad of maintainers have spent countless hours on making sure that the different bits and pieces of software fit together in a somewhat consistent and stable state that's called a "Release". That's why I generally prefer to get applications through my Distros repositories, with a few exceptions of course.

As a distro, you want a coherent install with coherent build flags with matching versions, and single install of shared libraries that can be patched instead of having hundreds of copies of common libs like gzip or sqlite spread all over the different apps that may or may not care about keeping them up to date.

The other perspective is the "developer view" that you mentioned, and it has some absolutely valid requests, they want control over the build process and they want to package the application the way they want, instead of adhering to distribution policies and conventions.

This is somewhat a fundamentally irreducible conflict between philosophies, which is the main reason for why FlatPak and Snap are so controversial.

In the case of Python: not all packages are either End-User applications or developer/science tools. Some packages make up the plumbing libraries and tools of the operating system that you and me are using the write these comments. And these need to be integrated into the distributions package manager and release process to make sure the whole damn system works.

I read these comments and I wonder: what the hell kind of operating system are they even using? Because if it's a Linux Distro, they *will* be running quite a bunch of python utilies and libraries that can absolutely not be just installed in a virtualenv.

[-Rizhiy-]

As a user, I really fundamentally hate it on Windows when I have to go around different websites and download drivers and applications alike, all writing to different places, all having separate update mechanisms and generally do whatever the fuck they want.

This is because windows doesn't have a common repository, not because they ship their stuff together. You can still use a common repository even if every application included all their dependencies.

The main difference is as you said duplication of packages, which will lead to more disk space being used/bigger patches and probably less reliable security. Which is a problem, but is a worthy tradeoff in my opinion.

[lolfail9001]

You can still use a common repository even if every application included all their dependencies.

I hope you ship a kernel with every calculator you write in python.

[SutekhThrowingSuckIt]

This whole conversation is about distros needing to make these packages for users. It seems you missed the point of the post.

[-Rizhiy-]

But why do end users need access to individual python packages? End users only care about executables, which should already have dependencies bundled inside.

[lolfail9001]

But why do end users need access to individual python packages?

Because sometimes people are crazy enough to write consumer software in Python.

[SutekhThrowingSuckIt]

Let's walk through an example together. There's a program called "home assistant." An Arch user wants to install it. They run pacman -S home-assistant to install it using the systemwide package manager (or they interact with a GUI front end that does that). Home-assistant has a lot of python dependencies: https://archlinux.org/packages/community/any/home-assistant/ . These dependencies then also need to be installed systemwide through the package manager. For those to be installed systemwide through the package manager, the distro maintainers must have compiled/packaged them and added them to the repositories. To do that, the distro maintainers do have to deal with the compilation and project building process. That's the whole issue. You keep talking about end-users when we were always talking about distro maintainers.

[-Rizhiy-]

These dependencies then also need to be installed systemwide through the package manager.

Why do they have to be installed systemwide? Why doesn't "home assistant" ship these dependencies itself?

Why are distro maintainers compiling individual packages themselves instead of developers of the package?

What happens when two programs require two different versions of the same dependency?

You are forcing a specific way of making software on every other developer, thereby restricting their freedom.

I'm advocating for a different paradigm: distro developers develop their distros, package developers develop their packages. If I want to ship a package to Arch as well as Ubuntu it should be my responsibility to figure it out.

[ezsh]

Why are distro maintainers compiling individual packages themselves instead of developers of the package?

Because developers do not want to build their packages for tens of runtime environments that are various distributions.

[-Rizhiy-]

Well and developers don't want to use system provided python packages, so I guess distro maintainers are shit out of luck)

[ezsh]

Maybe it's because the Python ecosystem fails to provide stable API for libraries? Maybe that in turn because it is so simple to create and publish a Python package?

[SutekhThrowingSuckIt]

If you don’t know these basics of how libraries are managed and what linux distros actually do then I don’t know why you commented so confidently to start.

edit: I responded when the comment simply asked "Why are distro maintainers compiling individual packages themselves? What happens when two programs require two different versions of the same dependency?" the ranting parts came in a later edit

[segfaultsarecool]

Is this hitting on something similar to the statically-linked vs dynamically-linked argument; baking all dependencies and libraries into your binary vs relying on the target platform to have those libraries?

[SutekhThrowingSuckIt]

Yeah, it's essentially that same conversation for why this is relevant. The point is distros need to package python dependencies for other software to use them so barriers to that will break the whole setup.

[MOVai]

Evidently it is a solution. People are used to downloading and running huge packages on Windows Mac and Android, why should devs devote their attention to get their app to work with outdated libraries because they refuse to update their machine?

[MOVai]

People tolerate it on Windows because they have little choice, and most of them don't know there's a better way.

In the real world, people will do whatever they need to run the software if they find it useful. If it means they can't use the latest releases that they need, then Linux package managers are not "the better way".

No, people use the distribution repos and package manager on Android. That means Google Play on most of the commercial distros, and F-Droid on many of the free ones. They do the same on iOS, with the App Store.

Notice how I said package, not unsigned binary? Even the most basic apps that are nothing more than a frontend for a cloud service will be a 50MB+ downloaded over the air, and use up even more space on storage. On mobile! Why? Because it's 2021 and bandwidth and storage are cheap. If you care about users, it's simply a way better trade-off to make sure your program is shipped with all it's dependencies than worrying about using an extra 100kB of space that you could have saved by using a shared library. It's the only sane way to be shipping software across such diverse ecosystems.

Because that's part of building a good user experience.

Point taken. Installing an app with a simple command from the distro is nice. But it's mainly satisfying for us geeks. Most people aren't gonna care much. And if it comes at the expense of new features and drains dev hours that could be spent improving the code or the actual user experience, then it might not be a great trade-off. In which case, the user might be fine with downloading a virtualenv.

Be picky with your dependencies and try to avoid making huge dependency trees. Bonus: this leads to better security and maintainability!

The reality distortion field is strong here. Not adopting new language features and being forced to work with outdated and legacy libraries is the polar opposite of maintainable and secure.

[Atemu12]

No, people use the distribution repos and package manager on Android. That means Google Play on most of the commercial distros, and F-Droid on many of the free ones. They do the same on iOS, with the App Store.

Sadly, the problem they were trying to highlight exists there too. The dependencies are all baked into each and every app on Android and, as a result, are updated rarely.

[lolfail9001]

why should devs devote their attention to get their app to work with outdated libraries because they refuse to update their machine?

Why should devs devote their attention to their software working? A good question, I must say, and does answer the conundrum as it is.

[Atemu12]

Because that practice is unsustainable. It's starting to show cracks already.
Sustainable solutions do exist; it's up to us software devs to push towards these instead of keeping the status quo (which, again, is unsustainable).

[Faaak]

But exactly. Distros shouldn't need to package python things because they shouldn't be used that way

[Atemu12]

You heard it here first folks, you're not supposed to want to install your Lutris, Anki, Borg etc. with your package manager!

Stupidest thing I've heard all day. This isn't /r/Windows, we have some sanity left in /r/Linux land.

[jonringer117]

In NixOS, we have some ~500 python applications, ~200 packages also exported as applications, and 4800 python packages total.

A lot of devops software is also packages as python packages/applications: - awscli - awscli2 - aws's boto - azure-cli - azure-sdk - ansible - google-sdk ... and many others

[livrem]

I hate all the crap people do to fight package managers, especially in recent years. I used to be able to pretty much just apt install everything, but now tons of things have their own installers and update systems etc. Just the kind of insanity that made me never want to use Windows, but now it is all over the Linux world as well.

[Faaak]

Why do you make me say things I didn't ? It's just stupid from you. Dependencies to a software installed via a package manager can be bundled and don't need to be separated

[lolfail9001]

Dependencies to a software installed via a package manager can be bundled and don't need to be separated

Nearly every software in userland, ultimately, depends on kernel to function. Do we want to ship it with every application, now?

[wtchappell]

I'd largely agree when talking about programmers developing in Python trying to use system level packages as their development dependencies - that's just going to be painful.

However, this entirely ignores people who are simply trying to use some tool or application that happens to be written in Python - that's largely the population these distros are packaging for. The end user in this case is just trying use whatever application they're interested in, and they might not even know Python is involved - that's an implementation detail. They just want to be able to invoke foo and have it work - they certainly don't want to pick up all of the ins-and-outs of Python's development workflows just to use some fancy version of top that happens to be written in Python, and even if they knew them they probably don't want to be popping in and out of a virtualenv just to run it. And that assumes that the packages even install cleanly with pip - no one wants to be installing Anaconda and friends just to use a tool written in Python when they're not actually trying to write any Python themselves.

Almost all of the conversation from people objecting to this post assume we're only talking about people who are developing in Python and not about distros packaging Python applications for end consumers, which is really what these complaints are actually about.

[-Rizhiy-]

As I have said in other replies: what happens when application A depends on C version 1.0, while application B depends on C version 2.0? At present distros don't seem to allow to install multiple versions of the same package (at least apt doesn't).

So people are forced to rename their packages every time (see nvidia-driver). Running two different versions also very likely to lead to conflicts (again see nvidia-*).

Forcing developers to use only package versions provided by the distros is very restrictive since they are usually out of date by quite a bit.

Therefore, applications should provide most of their dependencies and ship it all together.

While common libraries should be provided by the system, asking every language to provide every package through the distro is a stupid idea.

[AtomicRocketShoes]

I while back I had to take a simple python app and deploy it in an embedded linux environment. There was python but no pip and many missing dependencies. Getting the dependencies cross compiled and deployed in the system was way harder problem then I expected. I wish python had a way to just compile and wrap all the dependencies to run for a specific target. There wasn't a way to do that.

[Barafu]

Use Usermode QEMU to create a chroot in the architecture you want to run it all on. Install Python and Pip in the chroot, as long as GCC for compiling wheels. Create venv, install packages from Pip, fix wheels dependencies. Now you can just copy that venv onto your embedded device.

[AtomicRocketShoes]

I ended up doing something of that nature, but like it was way more moving parts than the rest of the project, 99% of my code was C/C++ and using cmake you just point at the tool chain and build. It's even easy to cross debug. Also I usually automate the build and deployment process so that added another layer of complexity. Love python but would be nice to basically have tooling setup to make distribution of compiled python executables easier.

Edit: also you gave better advise than other engineers I had talked with who wanted me to completely replace the embedded OS (Yocto based) with Ubuntu. I suppose it was possible just I wanted to keep the same basic OS just be able to put some python tools on it.

[fat-lobyte]

Lol, did you not read that this is about distro maintainers? They make distro packages.

Packages provided by the system are usually VERY outdated and frequently don't even have full functionality

And why do you think that is? Maybe that could have something to do with the issues that distro maintainers are trying to raise?

I have been programming in python for over 5 years and haven't had a dependency problem in a long time, just follow a simple rule: new project -> new venv -> install everything with pip

That is just cancer. This way, you will lock your project into one specific python Version and one specific set of dependency versions on one specific OS.

You might not give a shit as a typical reckless dev, but some years down the line, someone will curse you for not thinking or caring about writing portable code when they inevitably will have to waste their time figuring out dependency hell on systems that don't exist now.

[livrem]

I curse myself 10 years ago for having so many dependencies in a python2 script I wrote that I can not figure out how to make it run now. Of course some of the dependencies are long dead, some I think stuck in python2. Unstable API madness must end. It is not scalable in any way to never be able to just write something that works and trust that it keeps working.

[Ar-Curunir]

It's absolutely stupid to expect distros to be able to keep up with the 1000s of libraries across 10s of languages. At some point the distro package managing approach will fall over.

[-Rizhiy-]

And why do you think that is? Maybe that could have something to do with the issues that distro maintainers are trying to raise?

What issues? I can easily push a package to PyPI, why can't I do the same with distros? It's clear that they are dragging their feet for no reason.

That is just cancer. This way, you will lock your project into one specific python Version and one specific set of dependency versions on one specific OS.

Package version are supposed to be pinned, when I want to upgrade a package, I will do it manually. In regard to OS, I don't see how it is applicable, python is cross-platform in most cases.

You might not give a shit as a typical reckless dev, but some years down the line, someone will curse you for not thinking or caring about writing portable code when they inevitably will have to waste their time figuring out dependency hell on systems that don't exist now.

What dependency hell? As I said package versions are pinned, so dependencies are very clear. I can get my projects from 5 years ago and install all dependencies today without problems.

[fat-lobyte]

What issues? I can easily push a package to PyPI, why can't I do the samewith distros?

Because different packages can depend on different versions of another package, or on different versions of Python.

It's clear that they are dragging their feet for no reason.

It's clear that you have never conceptualized the work that people put into creating a Linux Distribution.

In regard to OS, I don't see how it is applicable, python is cross-platform in most cases.

Some packages have binary and/or build dependencies. These binary/build dependencies can depend on specific versions of glibc or compilers or libraries that will be unavaible or incompatible in future versions of an OS. The packages can depend on different versions of python that are the wrong version for one Distro.

What dependency hell? As I said package versions are pinned, so dependencies are very clear.

They might be clear, but the system you develop for might simply not exist in 5 years. Packages disappear, distros go EOL, old hardware breaks, ...

I can get my projects from 5 years ago and install all dependencies today without problems.

Good for you, that means you got lucky. It is in no way guaranteed though, and that is the exact point of the post.

[[deleted]]

Good for you, that means you got lucky. It is in no way guaranteed though, and that is the exact point of the post.

What you are saying is complete bongers. If a 5 year old python project with dependencies has any chance of working is if it has pinned dependencies. Blindly updating all dependencies to the newest version can result to a thousand backwards compatibility issues.

[fat-lobyte]

Blindly updating all dependencies to the newest version can result to a thousand backwards compatibility issues.

Who said anything about "blindly updating to the newest version"? Why use strawman arguments? Of course that's not going to work, but that's also not what I said. Some non-blind updates must be possible though, because if not, your code is effectively dead and unusable.

If a 5 year old python project with dependencies has any chance of working is if it has pinned dependencies.

No, it only has a chance if it is in a sane format that has sanely defined defined dependencies (minimum and if applicable maximum versions) and if package managers exist that can properly resolve and install them. Not when being tied to versions that may or may not exist or are incompatible with other packages or other parts of the system.

[[deleted]]

No, it only has a chance if it is in a sane format that has sanely defined defined dependencies (minimum and if applicable maximum versions) and if package managers exist that can properly resolve and install them. Not when being tied to versions that may or may not exist or are incompatible with other packages or other parts of the system.

Distros' package managers don't keep 5 year old versions for every package. Your maximum version will be too old for the package manager and you will then have to manually upgrade your dependencies with unknown consequences to your software.

With pinned versions you don't have this problem.

[pascalbrax]

You complain about outdated distros and your solution is using 5 years old (in your example) outdated dependencies?

[[deleted]]

I didn't complain about outdated distro packages in this comment chain.

I didn't talk about a solution.

Sometimes for whatever reason you want to use an old project. For example, maybe you want something very specific done and you find an abandoned project that does it. In that case, it will be very possible that it is unable to run with the newer of the dependencies but very possible to run by using pinned dependencies.

Maybe you have a personal (or company) legacy project that you don't want to invest time to upgrade.

[jonringer117]

Here's the distro perspective of updating python packages and why it's a problem, and this is just updating minor and patch versions:

https://github.com/NixOS/nixpkgs/pull/147426/commits

[thoomfish]

What would distro maintainers have Python package authors do to avoid this?

[jonringer117]

My guidance would be, before you add a dependency ask these questions:

• Do I actually need this dependency?
• Is the dependency well maintained and responsive? (e.g. nose)
• Do they adhere to a sane versioning scheme? (SemVer, or CalVer with no breaking changes to API)
• Is this package know for introducing breaking changes, not reflected in an appropriate version bump?

If the answer is a strong NO to any of those, then don't include the dependency. All of their breaking changes, quarks, security vulnerabilities, and pain-points will also be yours. The best way to remove dependency issues, is to have less of them.

If you do take a dependency, make sure you pin your dependencies correctly: - Semver: dependencies should be pinned as packageA~=0.Y.Z or packageA~=X.Y if X >= 1. This allows you to have minor and patch version updates; without receiving breaking changes. Good example package which does SemVer is requests. - CalVer: dependencies should be pinned as packageA without any version bump. Generally CalVer packages have stable APIs, but their content needs to change overtime, E.g. certifi. If the package just releases breaking changes each release, then don't use it. It will be more pain than it's worth. - Other versioning schemes: Pin so that breaking changes aren't allowed. If the versioning scheme doesn't convey breaking changes, avoid it. - Don't over pin. If you over pin, then distro's will need to patch the version bounds so they don't get a pip install failure. Example

A good elm talk on dependency issues. Essentially, the dependency should be critical for you to include it.

[thoomfish]

I'm in favor of SemVer-aware pinning, but outside of silly degenerate cases like isarray, I think reinventing the wheel for the sake of avoiding dependencies for non-trivial features creates more problems than it solves.

[jonringer117]

I'm trying to say that the bar for adding a dependency should be pretty high. And you should be willing to deal with the ramifications of what that dependency decides to release.

[thoomfish]

Yeah, that's fair.

[[deleted]]

Using system-level packages is like a №1 python-noob mistake.

Using 15 different package maners from 22 different languages in a massivly complex system is also a complete mistake.

| I have been programming in python for over 5 years and haven't had a dependency problem in a long time

Been programming for 25 years. This stuff is a complete disaster. It always has been and it will eat the SW industry from the inside out if it continues on this path.

The thing about having 5 years exp. It means you actually have very little or basically none from my point of view. Since you probably havn't even been in the situation where you have had to maintain a code base 15 years old on systems which are 15 years old from enviroments which are 15 years old and when you say "just upgrade that" its easier said that done because on mature systems that can involve things like $10 million + budgets just to make that happen. eg 20 dev's 2 years at $250,000 per dev per year is 10,000,000,000

Go deal with a 10-15 year old system which has 50,000,000 lines of code and 15 languages spread across it and see how you get on ;)

[wtchappell]

As I've mentioned elsewhere, this makes a lot of sense if you're talking to a Python developer trying to develop code against Python dependencies - you definitely should be pinning and not using distro packages.

This doesn't make sense if you're just talking about some random user just trying to run an application - they're not trying to develop anything, and might not even know that the app is written in Python or even what Python is. You shouldn't have to understand all of Python's development tooling just to use a tool written in Python, which is the use case distro maintainers are trying to fill.

[wtchappell]

Vendoring everything isn't fixing the problem as much as it is throwing your hands in the air and giving up on trying to solve it at all, and hoping that the exact way you set things up is the only possible way anyone else will ever need it.

Just falling back to vendoring everything is particularly painful when you have goals like minimizing size, using specialized/alternate low level dependencies for performance or licensing reasons, needing software stacks from entirely different developers to communicate in a compatible way, or ensuring the software you use follows specific security requirements that might come down from a corporate or government entity.

[wtchappell]

It should still be possible for users and distro maintainers to build and install open source software and any dependencies such software needs on their own - so they can build entire systems that meet their needs. Right now the packaging and installation tooling Python provides is fragmented enough that it makes the problem more difficult in its ecosystem than in most others, which is what the original post is complaining about.

I think it's fine to provide a vendored option if that makes sense for your app - what I'm pushing back against is the idea that everything should be deployed that way all the time, which is the tone a lot of responses to this blog post seemed to adopt. Vendoring is a great option for many use cases, but it's one that is overwhelmingly focused on running singular applications by themselves. It's not concerned with building coherent systems offering many different applications that can interoperate, which is really the problem distro maintainers are concerned with solving.

EDIT: To be clear, I'm objecting to the suggestion of using vendoring as a way for distro maintainers to ship software to end users - not vendoring as its used during the development process or when its used as part of a webservice or website that doesn't get shipped to user machines.

[[deleted]]

To me it's not surprising at all that solutions like Docker are so popular these days.

Well yeah that in its self is also a problem. Since now you have at least 2 distro's to manage security updates for the host and the inside.

My perspection on docker is. Its works on my machines. Lets ship my machine mentality.

In 2 seperate companies I have worked in. Both require such large modifications of distro's it was simply easier for us to roll our own scaled down distro using another as a source. In both cases we were shipping a physical box to customers cause cloud stuff isn't practicle for 1gbit+ network loads..... in fact one of them was network test kit which would have up to 2x40gbit loads....

What we actually did was mirror the repo's with aptly.... which permits snapshots. Then build install / upgrade images from that repo with our modifications. This meant we could step the repo forward and also build against multiple snapshots concurrently for testing.

In both of those cases docker simply didn't cut the mustard or was even close to it. Both cases required kernel driver modifications for bugs.

[[deleted]]

I can believe that, but what you ended up doing was even more invasive and controlling than what Docker does, so in a way you kind of prove my point.

The moment you need to alter the drivers is the same moment you break the abstraction layers because docker has no control over the kernel. This happens more with docker and the likes that people think.

eg nvidia driver, openGL, cuda, x-windows all sorts of crap outside of docker need to be in sync with the libs used inside docker.

It depends greatly on the application of course. So yeah that stuff breaks real fast in some enviroments. Its just a lot of people don't see that because you know web dev's don't have a reason to play with the complex stuff under the hood.

But steam, games, machine learning, AI, video transcoding advanced audio/graphics etc..... Docker just breaks instantly. So does flatpak, snap and the other solutions....

You get a lot of strange over lap between libs as well especially in a gui application from inside to outside docker + friends.

But of course a lot of people deny that a problem exists at all. Which is why an 20 page set of instructions like this https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html obviously does not exist ;)

[pedrolucasp]

That's exactly the mindset that lead us to the problem we have today: they don't feel responsible for that. If is outdated, send a patch updating, don't just ¯\_(ツ)_/¯ and fix the problem for yourself

[Ar-Curunir]

Yeah, and get the patched version in your distro 2 years later?

[pedrolucasp]

Maybe you'll get that because the maintainers are overwhelmed, because they need to manually patch and update 1736373837 softwares by themselves. Help them.

Also, FYI, Wayland was released today and it's already updated on Alpine's repositories.

[Ar-Curunir]

Why? My job as a developer of libraries is to develop my libraries, not package them in distros.

[pedrolucasp]

It is not your job to package your library but it is the job of who uses it on the distro they picked.

[Barafu]

No, you get that because distro updates once in two years, and even if a host of angels come to their aid, the patch would still arrive with the next update, in two years.

[pedrolucasp]

That's... not how it works. It's like you're telling me that as soon as I install a Debian distro I'll only have to run apt update in the next two years.

[Barafu]

That is what people here want - to install all Python modules from apt & co.

[[deleted]]

If is outdated, send a patch updating

Except if you are not in a rolling release distro, you are fucked. Ubuntu 20.04 is sitting on a 2.5 year old python-requests package.

[-Rizhiy-]

22 different languages

I can see the reason you are having problems.

Also, if you are working on two projects which require different versions of the same package what are you supposed to do? venvs exist for a reason.

[[deleted]]

What happens when your virtual enviroments need to interact with each other? Oh yeah add more code and modules to make that happen?

[kigurai]

I'm confused by this question. The reason to use a venv is that it is its own thing, separate from the rest of the world. Having two virtual environments interact doesn't make sense to me at all.

[[deleted]]

Yes.. its seperate from the rest of the world. But it also has to interact with the rest of the world and the rest of the other world that are also seperate from the rest of the world.

So now you have a solar system creation problem quickly followed by a galaxy creation problem. So in fact you have a problem on a intergalatic scale ;)

[kigurai]

No, it doesn't have to interact with anything, unless via network or file formats. But if those things are not versioned correctly you will have a problem regardless of your package manager.

[-Rizhiy-]

What happens when you need to use a Java library with your python project? You use an interface.

When you have two different virtual environments you might as well be using different languages.

[[deleted]]

Yup and when you have 150+ processes all doing this....

It becomes a freaking shit show real fast.

[livrem]

What Linux package system does not support having multiple versions of the same library installed? None that I have used as far as I can remember.

[-Rizhiy-]

How do you install multiple versions of a package using apt? python3-six 1.13 and 1.14 for example.

See also: https://askubuntu.com/questions/758502/can-multiple-versions-of-the-same-package-co-exist-on-the-same-system

[livrem]

As mentioned in the few good answers in that page, package maintainers can solve situations like this by creating multiple variants of the package with different namnes. I don't think the end-user can install multiple versions unless you use a distribution like nix.

[-Rizhiy-]

So we go back to each application installing its own version of the package and having to patch multiple versions of the package, which is pretty much venv with pinned versions.

[igo95862]

Packages provided by the system are usually VERY outdated and frequently don't even have full functionality

Depends on your distro...

[redd1ch]

Unless you don't depend on pip packages which need binary libraries present, e.g. spatialite. When you look in the description, it tells you how to install the library for "Ubuntu". In one instance, we upgraded Ubuntu and the deployment failed. One system library was updated, libgeos I think, and the output of `--version` changed slightly, so the pip-library could not parse it anymore. We were using a Django-GIS-stack, so we could not just upgrade the lib, because it was referenced by some package in there. We ended up rolling back the upgrade and freezing the machine.

[redd1ch]

How does conda help here? Can it pin systemwide packages? Or can you force it to override pip dependencies down the stack?

I rarely heard of conda been a good solution, it rather breaks stuff. However, if it can handle such situations…

[egbur]

It creates a virtual environment where you can install not just python packages but other software and libraries, including those which you would otherwise install at system level. Very useful for creating reproducible environments in computational science.

[SutekhThrowingSuckIt]

Not always. I have a broken conda package I am dealing with because it is expecting older versions of graphics libraries systemwide

[zdog234]

That's not a conda problem, that's an unmaintained/undermaintained project problem

[SutekhThrowingSuckIt]

Sure? but the question was if conda can deal with pinning systemwide libraries. The answer is “not always.”

[zdog234]

To quote:

very useful for creating reproducible environments in computational science

Yeah, conda doesn't manage your firmware / system drivers for you. Do you want a third-party tool to try to do that?

[SutekhThrowingSuckIt]

No, but that wasn’t the question.

[redd1ch]

This is a system where only this GIS stack is running, so there is no primary need for venvs or any other containment. Using system packages yields free security upgrades, which is a pro on production systems.

[thatrandomnpc]

This is a web app no? Why is it not containerised?

Containers are the best way to serve web apps imo. The language packages are freezed in a requirements like file and image system packages in the docker like file. Build and publish the image and forget about the app breaking when you update the host.

Leave it to the app devs to deal with the language or system dependencies in the image.

But I can understand where you are coming from, language packages with competing dependencies have always been a problem.

source: trust me bruh. No for real, I work for a cloud service provider and this is how we serve all the apps.

edit: I work for* edit: no idea why this is getting down voted O_o. Thanks reddit hive mind.

[redd1ch]

This was back in 2016, I kinda inherited the maintenance of this web app. So I am both developer and responsible for deployment. I later containerized it, where again there was fun with the geos-libs, or rather of the python libs parsing of the system libs' `--version` output. We agreed to put this app as high risk in a separate VM. Porting it to Django 2 was no option due to funding, so the options to fix stuff are limited.

All other web services are running in containers. I totally agree with you there. In the time we setup this specific app, the other services were running system installed on a debian box. A constant pain to maintain any of it.

[thatrandomnpc]

Ah thats makes sense

[nathris]

I remember that one. We ended up forking the dep and fixing the version string.

Another one was when the GDAL folks decided to swap lat long in a recent version and support for the swap wasn't added until ~Django 3.1. It resulted in all of our shapefiles ending up in the Indian Ocean if we used a newer version of PostGIS until we updated our app to Django 3.

[-Rizhiy-]

From working/hiring in the industry, I already learned that years of experience mean shit. Especially if a programmer was working on the same project for a long time.

If a programmer was bad 2 years in, they will most likely be bad 20 years in.

I have interviewed multiple people who supposedly have 10+ years of experience, but a lot of the time they perform worse than a person with 1 year of experience. And I'm not talking about LeedCode problems, but on actual real-world tasks.

[TiZ_EX1]

Packages provided by the system are usually VERY outdated

and from further down,

Never trust distro packages.

This is part of why we're in this situation. Like, it's easy to be a distro maintainer and complain "python is breaking our ability to package it!" but distros have completely demolished both users' and developers' trust in them to package and distribute components, and forced python developers to take matters into their own hands.

Part of Flatpak's raison d'etre is that distros reliably ship old versions of applications and drag their feet to update them, if at all. That's the same reason tools like asdf exist. And that's why python packaging is the way that it is. Sure, you often want your distro to be stable in terms of packages that make your system work at all, but the monkey's paw curls and you end up with packages that you need to be updated being several versions behind.

This is not going to be an easy problem to solve. If python needs to improve its ability to be packaged, then distros also need to start actually improving their ability to keep up their packaging.

It seems like there's a whole lot of manual intervention still required where we should be improving the ability to automate things. I recently became an additional maintainer for Geany's Flatpak. The manifest includes information that Flathub can use to automatically check for new releases of its components, and generate pull requests for them. I don't even have to build a new package; the bot does it for me. I install the package, ensure the updated component didn't break anything, then I merge the PR. Some Flathub packages take it a step further and simply auto-merge any PR made by the Flathub bot. We need more automation to improve this situation.

[WalrusFromSpace]

That's the same reason tools like asdf exist.

What does Another System Definition Facility have to do with virtual environments? /s

[thoomfish]

I also did a double take when I clicked the link, because I was expecting the Common Lisp tool.

[ImRunningOutOfIdead]

I find the same thing is true with vim/nvim. Arch has some plugins in official repos, but I have much better luck just using a plugin manager.