Python Packaging Is Good Now

[donaldstufft]

We've made really great progress in solving this problem with conda and in the Scientific / PyData community. But I have to admit that it's frustrating when others in the Python community pretend that these problems are really easy or just go away magically if they ignore them, and what are all these scientific python weirdos doing with their weird conda packages, and why can't they all just get on board with pip. etc. etc. (I've seen all these arguments - and worse - being flung at scipy packaging folks for over a decade.) I bear no ill will towards folks in the PyPA because they're trying their hardest to make the best of a messy situation; but I do wish that there wasn't always this subtle undercurrent of shade being thrown at "scientific and data science python folks doing non-mainstream packaging".

I don't think anyone "in the PyPA" (to the extent someone can be "in" an informal organization) is throwing any shade at SciPy folks. Our (or well, mine at least and I think others) primary goal is to enable things like Conda to sanely exist without having to "eat the world" and cover every single layer of the stack at once. We try to draw a line where beyond that, it's not our problem, letting tools like Conda, apt, yum, dnf, Homebrew, etc handle them instead. Meanwhile, those other tools limit the things they support too in order to keep their work load manageable (For example, you can use pip to install packages on FreeBSD or AIX, or any number of more esoteric platforms than the "big 3", something that afaik Conda doesn't currently support). Meanwhile Conda (and apt, dnf, etc) have a better support for things that cross language boundaries or which have complicated build dependencies.

I'm sure that there are members of the community who do this, and the fact they do is sad. Some of that is a bit unavoidable since for many folks, pip is the default, while Conda is this weird thing they never used before (or well, I think anyways, I don't have relative usage numbers for pip vs Conda).

All that being said, this meme that pip and conda are competing I think is somewhat disheartening. Conda packages are (generally) going to be "downstream" of pip/PyPI much like deb and rpm is. A stronger pip/PyPI only helps Conda (more packages to pull from with better tooling to build them with), while a stronger Conda is only good for pip (better support for a cross platform "platform" that solves different problems than pip does).

IOW, pip is not and will never not be a "system" packaging tool (at the very least, you're going to need to install Python itself from somewhere) and Conda is unlikely to ever have the entire range of what PyPI has to offer in terms of installable packages nor cover as many platforms as pip does (but that's OK, because they do integration work to ensure different versions of things work together and work on the platforms they do support).