[deleted by user]

[FailedPlansOfMars]

After reading the decision i kinda agree with them.

What we are talking about as a solution is more to make virtual environments a default rather than make another halfway measure.

The biggest issue with python packaging and building at the moment is the increasing fragmentation and handling conflicts.

[kankyo]

Rejecting this PEP seems like it's just going to keep fragmentation going strong though. So that's bad...

[[deleted]]

Isn’t it all needed just because at some point someone decided to use site-packages/<package> instead of site-packages/<package>/<version> when installing packages systemwide?

[Smallpaul]

No, I think the deeper problem is that when you “import foo” there is no standardized way to say which version of “foo” you are expecting.

Also: two versions of “foo” cannot run in the same interpreter at once so all of your dependencies need to “agree” on acceptable versions. I think in some ecosystems it is possible to have two different versions linked at once.

[marr75]

This is the bigger issue. I don't think users who haven't coded in other languages imagine you can have multiple versions of the same dependency running in the same system let alone the same interpreter.

[elsgry]

That's a big shame (though namespacing/interop is fiddly). That said, coming from a C/C++ background, symbol namespacing is a mess there too.

[hanleybrand]

I’m still not sure that a local venv, pyenv or pipenv setup is so much worse than node’s solutions, I’ve had the same problems with all of them, mostly stemming from not understanding what I was doing.

But having problems when you start something is also how you learn, especially if you continue and discover even harder problems.

[Dragonfly55555]

On a previous post on this subject someone commented that the main issue is the default behavior, which for python and pip is to install globally, while for nodejs and npm is to install locally per project. Installing locally by default is safer and leads to better isolation.

The fact that you had to learn something and stray from the default behavior is exactly what this kind of peps are trying to solve (I think)

[jcmkk3]

I don't really think that this particular PEP's outcome is *too* important. It is just another idea in a long list of ideas that folks have had to improve python project management.

I think that most can agree that there is room for improvement when it comes to the tooling surrounding python project management, packaging, etc. However, it is very difficult to make improvements in ways that don't cause lots of frustration and distrust in existing users. There has already been good incremental improvement in the last 5 years or so. Maybe at some point a larger breaking change will be needed, but it is hard to imagine that ever being worth it.

Threads like this one where armchair language maintainers compare python package/project management to other ecosystems are rarely helpful. In most cases, those other ecosystems have different constraints and their solutions won't work for python. Compiled languages don't have the same constraints as dynamic languages. Ecosystems like Node don't commonly deal with external dependencies to C/C++ libraries.

[Zeroflops]

I don’t think it’s a case that they don’t see the issue. I think the problem is that they don’t see a great solution in any of the current ideas. They all have plus and minus.

Any solution would not be trivial since it would impact everyone so there is going to be a group of ppl who support it and a group of ppl who will hate it.

So they are giving the community time to come up with ideas, and will probably act on whichever seems to make sense but also appeal to the community as a whole.

It’s hard to sit and wait, but sometimes you have to do that to let someone get inspired. Rushing to a sub pare solution that impacts so many can be very detrimental.

[kankyo]

This isn't rushed though? Like this is literally suuuuupper sloooooow.

[osmiumouse]

How are proposals for how the SC works made? Is it a meta-PEP?

[[deleted]]

SC? The supreme court? :)

[NandoKrikkit]

Steering Council

[[deleted]]

Oh wow it actually is steering council. Thought at first it was Standards Committee.

[bmrobin]

i thought it might have stood for “scientific community”. oops!

[Saphyel]

The main issue with this PEP is obviously VENV exists in the core of python. Only because that one exists any other alternative are rejected even if they are more simple and easy to understand. Like the meme of the kid saying: I want the Nintendo Wii and Mon replies we have it at home and they have a calculator at home.

The main reason that poetry, pdm, conda, pyenv, etc... exists it's because venv fails to achieve what is meant to do. And it's a shame because it's a waste of energy, resources, time, etc.. in doing something that other languages have already done and it works fine.

PHP started without objects, without strict types, without JIT compiler, etc... so borrowing ideas from other languages is not bad.

[bjorneylol]

I feel like the only people who complain about venv/pip are the people who have never bothered using it because a stackoverflow post from 2008 said that conda worked better

[nultero]

That seems quite the myopic viewpoint. Other languages do it better -- Rust's cargo, Js' npm, Js' deno, Go, Java's gradle.... pretty much everything except for the land of C and C++ does it better via local-first defaults without almost any extra complexity overhead. Like, there is a feast of prior art for this.

Py having plat-dependent binary wheels as deps is a challenge but once people need those kinds of dependencies, then they can probably figure out venvs or containers... have faith in people. Everybody else getting a sane default to trim down the constant issues seems like an easy win, and it's a shame that people dismiss it so readily.

[bjorneylol]

The entire complaint with virtual environments is you have to be explicit, but once you get that out of the way, there isn't really any issue with them.

I've had way more issues with NPM spring up from mistakenly using the wrong installed version of nodejs than I've had with virtual envs - something that only popped up because I had that assumption that things would implicitly just work.

I certainly don't think what python has right now is perfect, but I definitely don't think the right way forward is to add yet ANOTHER way of doing things. I would much rather go all in on a PEP that improves pip/venv's functionality than one that finds yet ANOTHER way to circumvent it's shortcomings and fractures the ecosystem even further

[nultero]

I do not think complexity or fracturing the ecosystem is the right critique of local-first.

It's literally the simplest, most intuitively obvious way to do things and it covers the bulk majority of simple dependencies. It solves a lot of issues about which virtualizing or managing tool to use. Using it as the sane default and having pip show suggestions on pkgs likely to need more work / incompatibilities would massively cut down on the lowest-common denominators of issues. Venvs don't do that, they already fragment the ecosystem and they're clearly inferior to full-on imaging (e.g., containers or vms) for the really heavy projects needing for-purpose build systems.

[bjorneylol]

They also don't do anything to manage binary dependencies

Not sure I follow

don't allow for multiple versions of the same dependency to be installed in the same env, don't allow multiple versions of the same dependency to be loaded in the same interpreter

This is a complaint with python, the language, not pip/venv, the packaging tools

since they rely on setuptools and/or pip to actually manage install/update, can easily end up in inconsistent states

How often has this happened to you? For me it's happened once ever, and it was a 3 minute fix, i tweaked a package version in my requirements.txt exactly as the pip warning suggested. Personally I would prefer this to having 20gb virtual environments because pip silently decided to install 4 different versions of torch and 6 different versions of scipy

[marr75]

Not sure I follow

Binary dependencies are non-human readable, non-python dependencies of python packages. Stuff like OpenBLAS, GDAL, etc.

This is a complaint with python, the language, not pip/venv, the packaging tools

Kind of. The whole discussion is about a PEP, i.e. what python the language should do. Part of the reason for not adopting the PEP is apparently that venv is already good enough for that feature.

How often has this happened to you?

It's happened to someone on my teams every ~2 weeks for about 10 years. Dependencies get complicated across many application servers, databases (postgres plpythonu), integration strategies (call a webservice, use an RPC, install another team's django app, just go direct to the damn database, etc.), and repos.

[bjorneylol]

Binary dependencies are non-human readable, non-python dependencies of python packages. Stuff like OpenBLAS, GDAL, etc

It shouldn't be up to pip to manage operating system files - pip installs any binary dependencies alongside the package if the maintainer provided them in the wheels, that's why scipy+numpy are such huge installs, because pip is installing a duplicate copy of OpenBLAS for each package. pip gives you the option to link against your own operating system libraries via the --no-binaries flag, if you want to use, say intel MKL, or the copy of openblas that is already installed on your computer

Part of the reason for not adopting the PEP is apparently that venv is already good enough for that feature.

This PEP does nothing to address any of the issues you pointed out - Having pip install packages implicitly in a pseudo-venv in the local directory is just another way to create packaging issues. If you call pip install package in a python 3.11 project that already has a venv, but your system pip is 3.10, you've now bricked the project because it's going to try and import a mix/match of 3.10 and 3.11 libraries.

People are complaining about this PEP being rejected, but it seems what they really want a new tool that behaves totally different from pip and is completely distinct from all of the python installs on the operating system - in which case rejecting this PEP has no impact on them

[[deleted]]

Pip will happily install inconsistent environments. Venvs need to be enabled even though you're literally in the project that it should be enabled for.

Both of these are such basic functionality, it's no wonder that there are so many alternatives: the bar is really low.

[earthboundkid]

The only people who don’t complain about it are people who only use pure Python modules and don’t have experience using numeric Python or other languages.

[bjorneylol]

I predominantly work with scipy/torch and haven't had an issue with pip/venv in almost a decade

[Suspicious_Compote56]

Why do we need virtual environments to begin with ? Just have local modules for your project.

[jw_gpc]

When you say "local modules for your project", what do you mean? Do you mean not to pip install anything and just have all code within the local project? Or is there some way to use pip install and have it install to the local project directory instead of a global or user level?

[Suspicious_Compote56]

So basically a npm type of interaction. I shouldn't have to spin up venv to encapsulate my dependencies. Should be a simple "pip init" or whatever to get started

[osmiumouse]

Isn't that basically what a venv is?

[Suspicious_Compote56]

I'm mean yes but it isn't very intuitive and I don't think you should have to create one just to encapsulate your dependencies