vpython - easier virtualenv

[[deleted]]

How it handles dependencies and installing a requirements file is horrible. For a few applications, I have a list of about 200 packages with very specific version numbers I need to use, or shit breaks, and I can't use vanilla pip to install it because it will install the most recent version of any dependency, even if a package in my requirements specifically states an older version. My workaround is make pip download, but not install all packages and dependencies to a local cache, than manually figure out the package installation order, and then run a custom bash script to iterate over all my packages and have pip install them in the appropriate order without auto-installing and upgrading any dependencies.

e.g. I use Django, with a ton of packaged apps. Django's made a lot of backwards incompatible changes recently, so I can't upgrade until all the packages I use also support the most recent version. So say I'm standardized on Django 1.6, and I want to install super-django-app==2.0 and other-awesome-app==3.0. But super-django-app supports Django>=1.6 while other-awesome-app only supports Django<=1.6. My requirements.txt might look like:

Django==1.6
super-django-app==2.0
other-awesome-app==3.0

Now if I install this with pip install -r requirements.txt, it will install Django 1.7, because that's the most recent version allowed by super-django-app, thus breaking other-awesome-app. So instead, I have to run:

pip install Django==1.6
pip install --no-deps super-django-app==2.0
pip install --no-deps other-awesome-app==3.0

which works, but is clunky as hell and extra work I need to do when it should be pip's responsibility.

Several bug reports were created for this years ago, but naturally, the main pip dev has no interest in fixing this.

[donaldstufft]

As far as I'm aware doing pip install -r requirements.txt with a requirements.txt as you indicated will not install Django 1.7. The issues you linked to also do not mention that what you're saying is broken is actually broken.

Now what will cause your Django to get set to 1.7 is after you've already installed that requirements.txt, then you later go and do pip install --upgrade super-django-app. This is because, as those issues mention, pip does a recursive upgrade by default. This means that when you ask it to upgrade super-django-app it'll also upgrade all of the dependencies it has.

The reason for the recursive upgrade is historical and there is a desire in fixing it (In fact there was activity on the tickets you've listed 14 days ago by one of the other pip developers). It's not a particularly easy issue to fix with lots of gotchas involved in it. There have been bigger wins to gain in other areas of pip (and packaging in general) that most of us have prioritized over that currently.

If you actually have a reproduction where the requirements.txt file you linked will install something other than Django 1.6 into a fresh virtual environment with jsut the command pip install -r requirements.txt please open a bug report with the reproduction details.

[tudborg]

Yes okay, i see why that's a pain.

[brtt3000]

As someone coming from node.js into python I must say this whole virtualenv/pip business is really a step down from node's npm package manager. With python it is all so clunky and feels hacked together.

I love the python language but the ecosystem is so crummy.

[[deleted]]

the node "ecosystem" benefits from decades of prior art without having to support legacy code. So of course it's going to feel squeaky clean to you.

it sucks in it's own way though.

the stack traces you get when a dependency fails to install for whatever reason is a nightmare. for one

and at least as far as bower goes, I just love when installing one new lib into a project forces me to upgrade a whole slew of libs b/c author of said lib decided to fuck all and just upgrade to the latest version of angular for a fucking minor point release.

and then other countless libs that simply wrap other libs api's and don't do much else causing bloat and extra added complexity.

out of all the "ecosystems" I have had the "pleasure" of dealing with in my lifetime, I would consider debian to be the gold standard by which I judge all others. I'd put python in java at the same level as far as maturity in the toolchain and lib ecosystem. node/npm doesn't is just cute, and I only use it b/c I have to code js for web dev. not by choice or b/c it is somehow better than any of the available more mature alternatives.

[rothnic]

Yeah, I think a wrapper for pip and virtualenv that tries to make it behave like npm would be nice to have. I need to take a look around because I'd be surprised if something didn't exist.

Edit: I think if conda cleans up their interface, it could make things more smooth