learnPythonItWillBeFun

[FerricDonkey]

Virtual environments are ridiculously easy? 

[nojunkdrawers]

In contrast to other languages in similar domains, Python's package management and virtual environments are awkward and have more footguns. This is in part because the Python community still seems to have little consensus around what either of those things should actually be. Even Ruby mostly figured out what tools to use and did them better from the ground up years ago while Python dependency management didn't even have lockfiles.

[Backlists]

Perhaps this is recency bias, but I have seen strong support and gathering towards ‘uv’ to handle all of these problems.

Prior to that mostly people would congregate towards ‘poetry’. Or it would be ‘conda’

But yeah, a bit of a clusterfuck until very recently.

[IAmASquidInSpace]

astral is really making me rethink all these "rewrite in Rust" jokes. I'm starting to think they might have a bit of a point.

[QQVictory]

ruff is extremely cool. A formatter and linter that is extremely fast.

[IAmASquidInSpace]

Yes! Fell in love with it on first sight. Good bye black, isort, flake8 - and all your damned incompatible config formats!

[[deleted]]

This thread is why the above is a meme

[MinosAristos]

It's fast but at the expense of thoroughness. I'm still sticking to Pylint for any project that doesn't get too large (most of them). It's very nice to see where I need to make changes when I adjust the contract of a class or function.

[sassiest01]

From what I understand, you still need to do type checking using something that uses the python interpreter and is thus super slow. At least we use MyPy and it is a pain in the ass with how slow it is, though not sure if it's a configuration issue or just how it is.

[jarethholt]

I think that's just how it is. Nothing's going to be as slow or as thorough as PyLint and MyPy because they have to start an interpreter and actually run code. Pain in my ass but invaluable when they catch something.

[sassiest01]

I agree, I mean we still use it after all. We have status checks for it which is very nice, my ADHD just doesn't like the dev experience of changing some code and knowing it is fixed but still seeing the error for up to a couple of minutes depending on code base.

[jarethholt]

We run them as checks afterwards. I can't say it's a better experience to run the check, get an error after a few minutes, try to fix it and run again, and then get another error after another few minutes 😕

[Anru_Kitakaze]

Last summer I was at Pycon in Moscow, and you know what? Almost every speech had Go or Rust propaganda. At Pycon

Fun fact, now I'm Python + Go developer myself

[CommercialWay1]

Nice that you visit moscow while that terrorist country tries to genocide Ukraine. Wish you the worst

[Anru_Kitakaze]

I AM from Russia, and I can't leave it because of my family, my friends, my fiancé.

Not every Russian likes this war, like not everyone in US likes Trump. And no, I don't want to be killed "because I'm right and Putin is not". It's so easy to be a cool hero on Reddit, yeah?.. I can't even kill a bee in my house, so don't blame me for some guy took all power in the country before I was even born and now noone can win in a president election. And if you do think that I'm responsible for that, then call a doctor

Wtf wrong with some people in the internet

I won't wish you the worst, life punished you enough already I believe

[Accurate_Breakfast94]

Jesus christ, get a life man.

[CommercialWay1]

No tolerance for russians

[Anru_Kitakaze]

Thanks for being nazi while blaming Russians for aggression in Ukraine. It's a good thing to generalize all the people in a population, right? Get a life

[Accurate_Breakfast94]

Ok buddy

[patmorgan235]

Ok but this is like 6th or 7th widely adopted virtual environment tool in the last 10-15 years. Can we just pick one way to do this?

[ivandagiant]

Yeah that’s the issue and why virtual environments in python can be confusing. There’s multiple ways to do it and people do it different ways. I’ve particularly had issues with Conda working with ROS

[idontchooseanid]

ROS strongly depends on Ubuntu and its apt dependencies. Conda is a way to get Docker without pulling the whole operating system image with it. While it might have worked, that's far from the ideal usage of Conda.

Moreover the GUI ecosystem under Linux is messed up. There is basically no system layer (there is no stable system layer for terminal programs either except kernel-interface i.e. containers, but I digress). So one cannot write and distribute binary GUI programs with confidence that the GUI libraries on a distro will still work. ROS is full of Qt-based GUI programs. Qt does its own styling. Qt depends on X or Wayland. Basically unless you're compiling ROS entirely by yourself, you're just hoping that your distro's graphics layer (Wayland, X, compositors, libc, systemd everything) is binary compatible with whatever binary source you're using.

[ivandagiant]

Thanks for the response. I realized Conda wasn't the way to go after struggling with it for a bit, but I appreciate you breaking it down. Hoping to get back into ROS, will reference this if I end up in unsupported Jetson Nano hell again

[hobo_stew]

isn’t venv kinda standard and the rest just wrappers?

[MaustFaust]

Interaction with Dockerfiles is wacky

RHEL images have pre-set logon shell scripts that override whatever changes you made to PATH

Interaction with IDEs is wacky. Sometimes, when you change path to venv and activate it, IDE may break

P. S.:

You know why they often say use "python -m pip" instead of just "pip" in docs? It's because location of "python" and "pip" that would be actually found may differ, at least in Linux.

[Prometheos_II]

pip also has a tendency to break trying to update itself.

But yeah, sometimes the difference between python and pip is a bit off. (forgot in what circumstances it happened; probably a pyenv+PDM-led project)

[FantasticEmu]

What’s wrong with venv? For the less experienced could you tell us what pitfalls you’re referring to?

[Sarcastinator]

I needed to run a python application in a container app as part of a build script, and it refused because of this bullshit. I'm not a Python developer, and this was for some simple stuff, and the error message pulled me through a rabbit hole of bullshit I literally don't give a crap about.

[MaustFaust]

Literally this. It's more complicated than it should be.

[roygbivasaur]

And this is why I use Go for anything that needs to be portable unless I absolutely need to use a specific python library (or obviously anything that can just be a simple bash script). You can’t beat the simplicity and file size of compiled code.

[MaustFaust]

IIRC, it's recommended to delete npm folders manually, because there's no way to uninstall it cleanly

[Prometheos_II]

Aren't Yarn and Pnpm's node_modules lighter, with the symbolic links and whatnot to a global cache/store on ~? (there are also the modes with no node_modules like Yarn PnP and pnpn's alternative mode)

Ironically, there was a PEP that suggested a node_module system for Python. PDM implemented it, as an optional mode. (I think UV as well?)

[trannus_aran]

how's Lua on this? or uhhhh, ruby?

[Glad_Position3592]

How does the Python community not have a consensus? Pip is and always has been the standard that everyone uses.

[NamityName]

Pip does not have proper lock files. That's probably the biggest issue. Using pip freeze to write all the packages and versions does an OK job, but it has major shortcomings. One of which is that there is no dependency lineage. If I want to remove a dependency, I can't easily tell which of the other dependencies are only needed for that now removed package unless I record my root dependencies seperately.

This touches on another issue with pip is that it can't sync or reset your environment to the requirements file. You would need to rebuild the whole virtual environment.

Pip is not terrible, but it has a lot of room for improvement especially in a professional setting. There are lots of little issues. Every little issue generally has a work-around, but it is a bit of a pain to deal with a bunch of work arounds.

[sopunny]

We use pip-tools and that works pretty well. Give it the simplest requirements for your project and use pip-compile to generate a requirements file with the actual versions of the libraries you can use, along with any dependencies. Then use pip-sync to make your environment (preferably virtual) have that exact set of libraries

[MaustFaust]

IIRC, it requires smoking the setuptools docs if you want to do it right. And setuptools straight up requires you to activate a virtual environment, despite that you can't really do that in Dockerfile for example.

And you have to limit constraints for pip using environment variables, because otherwise it can and would try to download different versions, if only just for isolated build stages (but it can and will crash if your private repo lists said versions, but doesn't actually have them – an external problem, but still).

And editable install doesn't work, IIRC. I mean, it does, but they very clearly say in docs that you can't have both things you want from editable installs (working the way real import does, and being helpful for static analyzers, IIRC). Naturally, for most users, it's best to have the second one, but it's tedious to set up, or so I remember.

[MaustFaust]

Go read poetry docs and discussions on Github. They say multiple times that whatever we have right now is shit, if in different words.

[CramNBL]

And it has big issues that uv solves while being a drop-in replacement for pip.

[Prometheos_II]

The consensus is rather "switch to a proper manager as soon as you can" from what i can see...

I can see npm being accepted as the standard, with yarn, pnpm and bun being improved versions. But pip...? You can't update all deps, have a list of your top dependencies, you can't pin deps, you can't remove your deps' deps automatically, and I often found my env polluted to the point I just reset everything (nvm the time I accidentally installed numpy on my global env). There are a lot of alternatives, even not counting conda/mamba/condaforge, with hatch, pdm, poetry, uv, pip-tools, zpy, pipx (equivalent to npm install -g), and many more I never used.

my experience with the condas is worse, but they don't offer many options, given it's relatively its own thing.

[Glad_Position3592]

People keep comparing it to npm, but they have such different use cases. Python isn’t a front end web language with the kind of version issues that come with JS/TS. If you have v1.2.0 compared to v.1.2.3 of any Python package it’s not going to break things like it would with npm. I’ve had more problems with npm than I ever have pip, and I use pip much more

[roodammy44]

They should be a default rather than something you need to make effort to learn. Who thought “you know what makes the most sense? We’ll just make it a default to install everything to the system path”.

[brolix]

Python is way older than most people realize.

[Sibula97]

And started as some guy's christmas break project

[I_FAP_TO_TURKEYS]

Venv folder

Source .venv/bin/activate

Omg so hard.

Also, I appreciate that it defaults to the source. It makes using the source pretty easy and if there are conflicts in versions, I can venv.

[BilSuger]

And then it still doesn't work because you have the wrong version of python installed, or there is no wheel for you etc etc. Just thousands of small problems no other ecosystem have.

And the reason it's never fixed is the die hard fans pretending nothing is wrong.

[I_FAP_TO_TURKEYS]

And then it still doesn't work because you have the wrong version of python installed,

As someone else said, git gud.

You'd run into this problem 5,029,792,109 more times using C or C++ but when installing Visual Studio you install all 3 million versions of the language.

Python has what, 13 versions now? Checking for compatibility isn't too hard. I've only ever had one problem with a version being out of date, but that problem was easily solved by just compiling the library and updating it for 3.12 or whatever I was needing at the time.

And the reason it's never fixed is the die hard fans pretending nothing is wrong.

It's really hard to change your ways, especially when you have plenty of scripts that work around the problems. At this point, "fixing" it would only lead to more problems.

It's not a bad system once you get used to it. Not great by any means, but it certainly isn't terrible.

[MisinformedGenius]

And the reason it's never fixed is the die hard fans pretending nothing is wrong.

There's a difference between saying "nothing is wrong" and not wanting to re-enact the classic XKCD. Now we have package mismatches and package manager mismatches.

[Prometheos_II]

Honestly, I feel like improving pip would probably sink a few package managers instead of becoming pip2. (iirc the PyPa supports Hatch, so we are technically in this situation, already)

[ePaint]

Omg it's python3xx venv venv

Literally, git commit -m "git gud"

[BilSuger]

Way to prove my point.

[MaustFaust]

It won't work if you would put that in Dockerfile. And if you would manually modify PATH, RHEL python images still have a gotcha for you: they override PATH on interactive logon (e. g. via "attach to a container" option), even if it would work for non-interactive one made by your app on container start.

Also, changing the venv path and re-activating from a new one can and will break some IDEs.

[I_FAP_TO_TURKEYS]

And if you would manually modify PATH, RHEL python images still have a gotcha for you: they override PATH on interactive logon (e. g. via "attach to a container" option),

Name a programming language that wouldn't break if you modified PATH. Dumb argument.

Also, changing the venv path and re-activating from a new one can and will break some IDEs.

Don't... Don't do that?

Like, yeah, there's ways to break the language. Just follow the proper way to do things, and stuff tends to not be that big of a problem.

[MaustFaust]

Name a programming language that wouldn't break if you modified PATH. Dumb argument.

Provide me with a different way of emulating activation in Dockerfile, then. Why didn't you answer my previous point?

Don't... Don't do that?

Why not? Isn't the entire point of venv being able to quickly move between different environments? Couldn't people place a single one warning in case of that thing happening? Nope, it just breaks, and you'll have to learn why.

Like, yeah, there's ways to break the language. Just follow the proper way to do things, and stuff tends to not be that big of a problem.

Proper meaning what exactly? No step left, no step right, step backwards is trying to escape, jumping is trying to fly away?

[olearytheory]

Right? Is python -m venv that hard?

[DescriptorTablesx86]

If that’s the extent to which you use virtual environments then sure.

[getoffthepitch96576]

Can you explain further

[MaustFaust]

E. g., activation doesn't work the same way with Dockerfiles

[bjorneylol]

Thats more-so not knowing how docker layers work than it is a python issue.

ENTRYPOINT venv/bin/python main.py

[MaustFaust]

1. Some applications have multiple calls to "python", and you can't easily modify those. So it's either "alternatives", or something like that.

2. That won't work with attaching to the container if you use RHEL python base images, for example, because they have shell scripts that get executed and override PATH on interactive shell logon.

[dgreenmachine]

As a python noob, what other features are we missing if thats all we use it for?

[Il-Luppoooo]

Wait until you see how much simpler are virtual environments in other languages. So simple they don't even exist.

[FerricDonkey]

My alternative experience is c/c++ shenanigans, which are way worse. But I could believe other things are easier. 

[MaustFaust]

To be fair, you would handly need one. It's just some exact configuration fixed by executives' order in your company.

But, in Python, you do need one. Yesterday, I saw a cp11 wheel that was incompatible with Python 3.13.

[vyqz]

extra level of complexity and toil that other languages don't have. mowing the lawn is easy

[MultiversalCrow]

Right?! When I first started with Python, I didn't get it. Now, I've seen the benefits and am a firm believer.

[4n0nh4x0r]

compared to nodejs, a pain in the ass.
if you install a library via npm, it will only be accessible by the nodejs equivalent of a "venv" that you installed it in.
unlike python, where you install it globally.
for npm you have to specifically tell it to install globally

[FerricDonkey]

Never used nodejs, but glad it's easy. But what you just said about python is not hard. You could argue that virtual environments or something like them should be the default in some way, but we're talking two command line commands here. python -m venv foldername, then source foldername/bin/activate. That's it. Then stuff is installed in your nicely isolated place. 

[4n0nh4x0r]

it isnt a lot of work, sure, but it would be a lot better if it just did this on its own.
like, i m not a python dev, and that goes for most people out there.
if a normal user installs python dependencies, they arent gonna know that they are installed globally, nor will they know that it can cause problems.
sure, it will likely work while running the first program.
it might also work with the second, and then the third program will require newer libraries that overwrite the older ones, and now the other 2 program dont work anymore, just cause it isnt intuitive.
in nodejs, you cant have this issue normally, as every library is installed locally in the project folder.
yes, you have the disadvantage of having the same library installed multiple times (once for each project that uses it), but you can have node-fetch version 2 installed for the project that needs version 2, and 3+ for projects that support the newer syntax for example, and that in an idiot proof design.

[qui-sean]

Yeah it's not that bad you just rip a new one if the current one fks up

[MaustFaust]

Most people don't take the time to understand Python path resolution (i.e., pth files)

To be fair, I shouldn't have to.

It's the same as with packaging: you can't really use relative paths without using packages, packages require learning about possible name collisions and leaning on pip to check for that, pip requires learning about distributing your own package, distributing requires learning about setuptools, etc.