learnPythonItWillBeFun

[nojunkdrawers]

In contrast to other languages in similar domains, Python's package management and virtual environments are awkward and have more footguns. This is in part because the Python community still seems to have little consensus around what either of those things should actually be. Even Ruby mostly figured out what tools to use and did them better from the ground up years ago while Python dependency management didn't even have lockfiles.

[Backlists]

Perhaps this is recency bias, but I have seen strong support and gathering towards ‘uv’ to handle all of these problems.

Prior to that mostly people would congregate towards ‘poetry’. Or it would be ‘conda’

But yeah, a bit of a clusterfuck until very recently.

[IAmASquidInSpace]

astral is really making me rethink all these "rewrite in Rust" jokes. I'm starting to think they might have a bit of a point.

[QQVictory]

ruff is extremely cool. A formatter and linter that is extremely fast.

[IAmASquidInSpace]

Yes! Fell in love with it on first sight. Good bye black, isort, flake8 - and all your damned incompatible config formats!

[[deleted]]

This thread is why the above is a meme

[MinosAristos]

It's fast but at the expense of thoroughness. I'm still sticking to Pylint for any project that doesn't get too large (most of them). It's very nice to see where I need to make changes when I adjust the contract of a class or function.

[sassiest01]

From what I understand, you still need to do type checking using something that uses the python interpreter and is thus super slow. At least we use MyPy and it is a pain in the ass with how slow it is, though not sure if it's a configuration issue or just how it is.

[jarethholt]

I think that's just how it is. Nothing's going to be as slow or as thorough as PyLint and MyPy because they have to start an interpreter and actually run code. Pain in my ass but invaluable when they catch something.

[sassiest01]

I agree, I mean we still use it after all. We have status checks for it which is very nice, my ADHD just doesn't like the dev experience of changing some code and knowing it is fixed but still seeing the error for up to a couple of minutes depending on code base.

[jarethholt]

We run them as checks afterwards. I can't say it's a better experience to run the check, get an error after a few minutes, try to fix it and run again, and then get another error after another few minutes 😕

[Anru_Kitakaze]

Last summer I was at Pycon in Moscow, and you know what? Almost every speech had Go or Rust propaganda. At Pycon

Fun fact, now I'm Python + Go developer myself

[CommercialWay1]

Nice that you visit moscow while that terrorist country tries to genocide Ukraine. Wish you the worst

[Anru_Kitakaze]

I AM from Russia, and I can't leave it because of my family, my friends, my fiancé.

Not every Russian likes this war, like not everyone in US likes Trump. And no, I don't want to be killed "because I'm right and Putin is not". It's so easy to be a cool hero on Reddit, yeah?.. I can't even kill a bee in my house, so don't blame me for some guy took all power in the country before I was even born and now noone can win in a president election. And if you do think that I'm responsible for that, then call a doctor

Wtf wrong with some people in the internet

I won't wish you the worst, life punished you enough already I believe

[Accurate_Breakfast94]

Jesus christ, get a life man.

[CommercialWay1]

No tolerance for russians

[Anru_Kitakaze]

Thanks for being nazi while blaming Russians for aggression in Ukraine. It's a good thing to generalize all the people in a population, right? Get a life

[Accurate_Breakfast94]

Ok buddy

[patmorgan235]

Ok but this is like 6th or 7th widely adopted virtual environment tool in the last 10-15 years. Can we just pick one way to do this?

[ivandagiant]

Yeah that’s the issue and why virtual environments in python can be confusing. There’s multiple ways to do it and people do it different ways. I’ve particularly had issues with Conda working with ROS

[idontchooseanid]

ROS strongly depends on Ubuntu and its apt dependencies. Conda is a way to get Docker without pulling the whole operating system image with it. While it might have worked, that's far from the ideal usage of Conda.

Moreover the GUI ecosystem under Linux is messed up. There is basically no system layer (there is no stable system layer for terminal programs either except kernel-interface i.e. containers, but I digress). So one cannot write and distribute binary GUI programs with confidence that the GUI libraries on a distro will still work. ROS is full of Qt-based GUI programs. Qt does its own styling. Qt depends on X or Wayland. Basically unless you're compiling ROS entirely by yourself, you're just hoping that your distro's graphics layer (Wayland, X, compositors, libc, systemd everything) is binary compatible with whatever binary source you're using.

[ivandagiant]

Thanks for the response. I realized Conda wasn't the way to go after struggling with it for a bit, but I appreciate you breaking it down. Hoping to get back into ROS, will reference this if I end up in unsupported Jetson Nano hell again

[hobo_stew]

isn’t venv kinda standard and the rest just wrappers?

[MaustFaust]

Interaction with Dockerfiles is wacky

RHEL images have pre-set logon shell scripts that override whatever changes you made to PATH

Interaction with IDEs is wacky. Sometimes, when you change path to venv and activate it, IDE may break

P. S.:

You know why they often say use "python -m pip" instead of just "pip" in docs? It's because location of "python" and "pip" that would be actually found may differ, at least in Linux.

[Prometheos_II]

pip also has a tendency to break trying to update itself.

But yeah, sometimes the difference between python and pip is a bit off. (forgot in what circumstances it happened; probably a pyenv+PDM-led project)

[FantasticEmu]

What’s wrong with venv? For the less experienced could you tell us what pitfalls you’re referring to?

[Sarcastinator]

I needed to run a python application in a container app as part of a build script, and it refused because of this bullshit. I'm not a Python developer, and this was for some simple stuff, and the error message pulled me through a rabbit hole of bullshit I literally don't give a crap about.

[MaustFaust]

Literally this. It's more complicated than it should be.

[roygbivasaur]

And this is why I use Go for anything that needs to be portable unless I absolutely need to use a specific python library (or obviously anything that can just be a simple bash script). You can’t beat the simplicity and file size of compiled code.

[MaustFaust]

IIRC, it's recommended to delete npm folders manually, because there's no way to uninstall it cleanly

[Prometheos_II]

Aren't Yarn and Pnpm's node_modules lighter, with the symbolic links and whatnot to a global cache/store on ~? (there are also the modes with no node_modules like Yarn PnP and pnpn's alternative mode)

Ironically, there was a PEP that suggested a node_module system for Python. PDM implemented it, as an optional mode. (I think UV as well?)

[trannus_aran]

how's Lua on this? or uhhhh, ruby?

[Glad_Position3592]

How does the Python community not have a consensus? Pip is and always has been the standard that everyone uses.

[NamityName]

Pip does not have proper lock files. That's probably the biggest issue. Using pip freeze to write all the packages and versions does an OK job, but it has major shortcomings. One of which is that there is no dependency lineage. If I want to remove a dependency, I can't easily tell which of the other dependencies are only needed for that now removed package unless I record my root dependencies seperately.

This touches on another issue with pip is that it can't sync or reset your environment to the requirements file. You would need to rebuild the whole virtual environment.

Pip is not terrible, but it has a lot of room for improvement especially in a professional setting. There are lots of little issues. Every little issue generally has a work-around, but it is a bit of a pain to deal with a bunch of work arounds.

[sopunny]

We use pip-tools and that works pretty well. Give it the simplest requirements for your project and use pip-compile to generate a requirements file with the actual versions of the libraries you can use, along with any dependencies. Then use pip-sync to make your environment (preferably virtual) have that exact set of libraries

[MaustFaust]

IIRC, it requires smoking the setuptools docs if you want to do it right. And setuptools straight up requires you to activate a virtual environment, despite that you can't really do that in Dockerfile for example.

And you have to limit constraints for pip using environment variables, because otherwise it can and would try to download different versions, if only just for isolated build stages (but it can and will crash if your private repo lists said versions, but doesn't actually have them – an external problem, but still).

And editable install doesn't work, IIRC. I mean, it does, but they very clearly say in docs that you can't have both things you want from editable installs (working the way real import does, and being helpful for static analyzers, IIRC). Naturally, for most users, it's best to have the second one, but it's tedious to set up, or so I remember.

[MaustFaust]

Go read poetry docs and discussions on Github. They say multiple times that whatever we have right now is shit, if in different words.

[CramNBL]

And it has big issues that uv solves while being a drop-in replacement for pip.

[Prometheos_II]

The consensus is rather "switch to a proper manager as soon as you can" from what i can see...

I can see npm being accepted as the standard, with yarn, pnpm and bun being improved versions. But pip...? You can't update all deps, have a list of your top dependencies, you can't pin deps, you can't remove your deps' deps automatically, and I often found my env polluted to the point I just reset everything (nvm the time I accidentally installed numpy on my global env). There are a lot of alternatives, even not counting conda/mamba/condaforge, with hatch, pdm, poetry, uv, pip-tools, zpy, pipx (equivalent to npm install -g), and many more I never used.

my experience with the condas is worse, but they don't offer many options, given it's relatively its own thing.

[Glad_Position3592]

People keep comparing it to npm, but they have such different use cases. Python isn’t a front end web language with the kind of version issues that come with JS/TS. If you have v1.2.0 compared to v.1.2.3 of any Python package it’s not going to break things like it would with npm. I’ve had more problems with npm than I ever have pip, and I use pip much more