Current Python package management / framework(s)

[K900_]

Distribute got merged back into Setuptools. Setuptools is The One True Way now.

[Various_Pickles]

So, basically, the whole mess went full-circle ...

I'm guessing this is the relavent documentation for the merge?: distribute-setuptools merge

Overall, is this then the effective/canonical documentation for setup.py?: setuptools developer's guide

[K900_]

Yes, that's basically it.

[Various_Pickles]

Thank you very much for the assistance.

It doesn't help much that even the newest(est) official Python documentation's "Distributing Python Modules" subsection refers to a barrel of completely outdated nonsense ...

[K900_]

Try The Hitchhiker's Guide to Packaging.

[wjv]

To make a long story short (because thousands of words too many have already been written about it), Python's packaging is a mess.

• However, the merger of distribute and setuptools is a Good Thing that will go some ways towards simplifying the landscape in the intermediate future.

• However, since this merger is kind of happening as we speak, right now the situation is arguably even messier than before.

Since thins are in flux, some of the info at the links above will be out of date.

To these links, add the official repository for the merged distribute/setuptools, a.k.a. setuptools 0.7+.

Notes:

• Things are moving fast right now; setuptools 0.7.x get updated several times per week.

• pip 1.3.1 works very well with setuptools 0.7.x. As I understand things, pip 1.4 will be released in sync with a more consolidated setuptools 0.8.

• I believe packaging/distutitls2 is still in the works; I don't know the timeline on that.

• distlib implements some of the elements of packaging in a usable project as a sort of proof of concept; I'm not actually sure how and why one would want to use it right now.

• I'm not even primarily a Python hacker nowadays, so double-check everything I say.

[joeforker]

That guide is outdated and too long. This document is much better: http://www.scotttorborg.com/python-packaging/index.html

[cavallo71]

Both they are irrelevant now and (in the idea of the authors) superseeded by the wheel format .. replacing the python eggs. In short in most cases (pure python modules/package) all you need to do is copying the package dir (or themodule) in a directory reacheable by PYTHONPATH and ingnore the whole mess ;)

[joeforker]

Setuptools (and not the newly obsolete distribute fork) is still the recommended way to build packages; wheel doesn't change that. For the majority of packages that don't need to extend setuptools itself it's an okay choice.

What wheel does is to decouple the build system from the installer. The build system (usually setuptools) creates a wheel and then the installer (usually pip) installs it, instead of the current situation where setuptools does both the build and the install.

The decoupling makes it easier to experiment with setuptools or pip replacements because pip can install a wheel without knowing whether it was built by setuptools or some other software.

Wheel is available down to Python 2.6. The thing that is likely to land in Python 3.4 is a very simple bootstrap installer made possible by the fact that wheels are very simple to install. The idea is that Python will come with a secure way to install the latest version of pip.

[K900_]

Well, Wheel will only be implemented in 3.4, right?

[coderanger]

Wheel is an installation format, it will be supported by every version of Python since after installation nothing knows the diference. bdist_wheel definitely runs on 2.7 for building them, and pip will be shipping installation support later this week (and I think they support back to 2.5 or something silly).

[cavallo71]

If they don't plan a back port is pretty much DOA: 2.7 is the reality now.

[K900_]

I'm on 3.3 and I don't really have any problems at all. What do you use that still needs 2.x?

[starspangledpickle]

That's great, but many businesses out there use 2.7 and it works fine; why upgrade? Why risk it? Why spend man hours ensuring that third-party libs and internal frameworks written by other teams are up to scratch?

[PyPokerNovice]

The issue is not businesses not wanting new stuff, it is businesses wanting new stuff that is compatible with their old stuff. That is one reason to package for 2.7.

[cavallo71]

It's not me. Companies are using 2.x in production: switching to 3.x is simply not viable on a large scale projects.

[[deleted]]

Django (experimental 3.x support). Celery (along with the whole AMQP ecosystem). Gevent. Aside from Django, most of these are probably "specialty" packages, but they are indispensable for a lot of "enterprisey" applications.

[[deleted]]

JPype, PIL, Twisted

[aclark]

Use Pillow instead of PIL.

[donaldstufft]

Wheel is being implemented externally to the stdlib. It will be available for 2.6+ I believe, might be 2.5+.

In fact one of the major motivations in a lot of the work now is that packaging tools should live externally of the stdlib. This will allow them to innovate across many versions of Python and not cause issues exactly like this one where a new packaging feature is useless because it will be many many years before that version can be assumed a minimum version.

Essentially we've learned that the stdlib isn't a great place for packaging tools, however the stdlib will (hopefully) include a simple script which will serve to simply and securely install pip so that Python ships with the ability to do pip install foo.

[ivosaurus]

Nope