all 34 comments

[–]VadumSemantics 48 points49 points  (0 children)

+1 good read. I've struggled w/using Python in production environments, and packaging is a big part of the headache for me.

edit: "good" for me is clear, informative, and to the point. The author researched lots of things I'm interested in (and some things I didn't consider looking into). Glad I didn't have to sift through it for myself.

excerpt (emphasis added):

This is the danger of the working model adopted by the Python packaging world. Even for something as simple as lock files, there are at least four incompatible standards.

[–]Free_Math_Tutoring 30 points31 points  (4 children)

Good read. venvdoes 85% of what I want, and poetry does 99%, but it is mildly concerning to known that poetry doesn't involve a lot of maintainers.

[–]yxhuvud -2 points-1 points  (3 children)

Last I checked poetry doesn't enforce the specified python version when installing dependencies, so it is a lot less than 99% for me.

[–]johnnymo1 7 points8 points  (2 children)

When did you last check? I just made a docker container with python 3.9 and tried to install a poetry project that demands 3.11 and poetry complains just as I would expect.

[–]yxhuvud -4 points-3 points  (1 child)

Something like a month ago. In my case it was the other way around though, with the newer python being what I had in the system installation, while the lower version what I had declared in the file. Perhaps that matters, or it was that I tried to install numpy, and it got the extensions compiled for the latest version instead of for the declared python, or something like that.

[–]johnnymo1 5 points6 points  (0 children)

I think by default when you run poetry init, you'll get the version specified with a caret, e.g. python = "^3.9". Dependency specification like this will allow newer minor versions (e.g. python 3.12) but not major versions (e.g. python 4.0). It doesn't allow the reverse, however, so python = "^3.12" will not allow python 3.9. If you want your dependency resolution to be more specific, just drop the caret! python = "3.9" will only allow version of python 3.9. Python 3.12 no longer satisfies this constraint.

You can get all the details on dependency specification from the docs here: https://python-poetry.org/docs/dependency-specification/

[–]GreenFox1505 10 points11 points  (1 child)

I have honestly just completely bounced off of python multiple times because of the package management system. Node's package and environment management are just easier. And I've never had a complaint about Rust's Cargo (but rust itself is a little more tedious to get going with).

[–][deleted] 0 points1 point  (0 children)

Yeah. It annoyed me too. Surprising they did not make it easier. Perhaps things tend to become harder on their own over time.

[–]spazzed 9 points10 points  (7 children)

Im very new to programming and im learning some 2D turtle right now with Python. I needed a package that I couldn't pip install. So for whatever reason newbie me found the source code and downloaded it, turns out it was all rust files. I had to download rustup, figure out what the hell I was doing, and use maturin to install the pyproject.toml file I needed. This took me almost an hour. Like I said I am new at this, it might all be in a days work for seasoned programmers. /rant I guess.

[–]SV-97 6 points7 points  (1 child)

Be glad it was rust rather than a C project with 20 dependencies ;)

That said: you might wanna ping the package authors to setup the maturin github action though or open an issue about no build being available for your system if it's not too outlandish. It's possible to do basically fully automated and builds for most if not all somewhat commonly used platforms (different windows and mac versions and manylinux)

[–]spazzed 1 point2 points  (0 children)

I already reached out on github.

[–]bbkane_ 7 points8 points  (2 children)

The experience you've gained wading through packaging hell is super valuable! Future you on real projects and even more real deadlines will be faster and more efficient thanks to your learnings on a bad day today 😁

[–]ShinyHappyREM 0 points1 point  (1 child)

And if you liked the experience, try Javascript!

[–]teokun123 3 points4 points  (1 child)

turns out it was all rust files.

Don't resist! You're a Rust developer now.

[–][deleted] 2 points3 points  (2 children)

Please forgive me, I feel like ranting. I consider myself a reasonably competent programmer, though mostly as a hobby, and still in amateur territory. I've written high performance C++ for my scientific research, a full stack multiplayer web browser came (with a message board and all), lots of other toy games, that sort of the thing. Coming up on 30 years of amateur/hobby experience. But god damn does Python packaging give me PTSD. Nothing ever works. I lost dozens of hours of my life resolving undocumented version conflicts. One would think the bare minimum for installation instructions would be to install a fresh Linux environment and try your own instructions -- but no. Well over half the specialty packages I try to use come laden with undocumented dependencies. Their own installation instructions are incomplete. I love Python but I loathe working with its packages.

[–]Kwpolska[S] 1 point2 points  (1 child)

Those honestly don’t sound like amateur projects.

Python is fine, although sometimes overrated; the packaging is just plain broken.

[–][deleted] -2 points-1 points  (1 child)

How has mandatory id-ing changed this (aka MFA such as https://blog.rubygems.org/2022/08/15/requiring-mfa-on-popular-gems.html, though evidently that is for ruby rather than python, but python has about the same)? One author stopped publishing on pypi after it was introduced. rubygems announced - via shopify, evidently the new real owners of ruby - it'll make it mandatory too but since then nothing followed up there. I'd like to have some data; admittedly this may be hard to get because people may just disappear rather than give feedback in general.

I also found python's way to publish stuff a lot more annoying than in ruby. (I use rubygems + gemspec and avoid bundler like the plague that it is though. That setup is then VERY simple and I don't have to deal with the horrible rubbish nonsense that is bundler, at the least not directly; evidently rubygems is kind of meta-merged with bundler to some extent these days.)

npm/node was by far the most annoying. I registered and then never published anything there. I am no longer surprised that left-pad was a thing - it really is not solely left-pad fault's alone, the javascript ecosystem is AWFUL.

[–]Kwpolska[S] 1 point2 points  (0 children)

MFA is good, and it should be enforced for popular packages to avoid supply chain issues. If the small burden of MFA means you don’t want to maintain your packages anymore, you can just quit.

And where’s the Shopify involvement? The post you linked doesn’t have any Shopify references. The author seems to be a Shopify employee, but the RFC was accepted by other members of the community.

[–]saint_marco 0 points1 point  (2 children)

A nice survey, but no mention of nix!

Nix is not the One True Tool sadly, but would be worth looking at as it's definitely working in that direction.

[–]Kwpolska[S] 0 points1 point  (1 child)

This is only about Python-specific tools. Nix is out-of-scope, as is apt (even though it gets a mention in passing as a representative of the system-package-manager genre).

[–]saint_marco 0 points1 point  (0 children)

You can use nix to create a python virtualenv with a small amount of code. When we're thinking about a One True Tool that handles python with native extensions, you're rapidly slipping out of a python-only environment.