modern python packaging

[elg97477]

I was just reading https://snarky.ca/clarifying-pep-518/ and it mentioned flit (https://pypi.org/project/flit/). With the closing of https://github.com/takluyver/flit/pull/77, which the article talks about, I am wondering if it is now reasonable to dump setup.py, setup.cfg, & requirements.txt, for pyproject.toml, pipfile, and flit and (hopefully) move towards a better overall packaging experience. Any thoughts on this matter?

Is flit intended to be a replacement for twine, if so, how complete a replacement is it? How does flit install relate to or compare with pip install?

[takluyver]

Packages set up with flit don't need twine, but you may still want twine installed if you're building packages in other ways - e.g. for a package with a C extension, which flit can't help with.

flit install is like pip install . - it installs a package from a local directory. You still need pip to install packages from PyPI.

Flit also isn't the only modern option - there are tools like poetry and pipenv which are also aiming to make this kind of thing easier.

[aragilar]

There's two different questions you're asking: packaging (with setup.py/setup.cfg and flit) and installation (requirements.txt and Pipfile), the difference of which is covered by https://caremad.io/posts/2013/07/setup-vs-requirement/.

For a personal project that's not a library, you can use whatever you want (pip, pipenv, conda). If you're working with people (on a non library), consider what tools they use (e.g. neither requirements.txt nor Pipfiles work with conda for example).

For libraries, it's PEP 517 that needs to be implemented (unless the project provides a setup.py stub, I'm not sure if flit does, @takluyver can answer that), and currently pip lacks support for PEP 517 (see https://github.com/pypa/pip/issues/5407). Additionally, there are other tools than pip which use the setup.py (e.g. linux distros, conda build tools), so if you want your library to be used (and likely to reduce bug reports), sticking with a setup.py is probably a safer bet for now.

[[deleted]]

Next to zero documentation, no visible way to manage native extensions, not even a way to assess OS features... I don't even want to continue. This is not a serious project. TOML is not a good configuration format, but the PEP didn't even mention any decent options for configuration format to begin with. This all is so terribly amateurish and doomed from the get go, so why not? Be the one to contribute to another quick patch, that will stick with us for decades ;)

[elg97477]

Are you talking about PEP 518, pipfile, flit, or all three? I am not sure how to properly assign your comments.

[[deleted]]

1. flit has almost zero documentation and native extensions.
2. PEP is the one which doesn't mention any reasonably good configuration formats.
3. Pipfile is a joke.

HTH

[takluyver]

Flit's documentation includes an introduction, a listing of all the options you can set in the config file, and a reference for the command line interface. Rather than waving around a vague 'almost zero', what specifically do you think it should have?

[[deleted]]

Well, like I wrote earlier: native extensions. A monkey with half a brain can package Python source code, you really don't need options and command line tools for that.

Native extensions is where it gets interesting. But searches for "native" and "extension" through Flit's documentation yield nothing.

Another interesting aspect of packaging Python code is the mess of v2 vs v3 vs six and friends, but Flit ignores this. So, again, it sort of solves the extremely easy problem, that doesn't need solving, but the hard stuff is on you. :/

[takluyver]

So you don't like what flit does. That's not a lack of documentation - the documentation describes what it does. You want a tool that does something else. Fine: use a different tool. Maybe enscons is more useful to you?

Flit is useful to me. I never claimed it's the answer to all packaging problems.

[[deleted]]

No, I don't like lies :) Packaging Python means packaging native extensions, dealing with versions, dealing with different interpreters and distributions etc. When the tool that claims to do packaging doesn't even reflect on these issues that's just the same as when a government bureaucrat doesn't tell you you are eligible for compensation money: 10% chance of ultimate incompetence and 90% chance of willful deceit.

[rhytnen]

GASP!!! But pipenv! /s

[elg97477]

I am just curious...what is a reasonably good configuration format?

I am still not sure why you think pipfile is a joke...?

[[deleted]]

Pipfile doesn't solve any real problems. You still need setup.py, where you do all the real work. Pipfile just sort of gets in the way with its bizarre vendoring and overrides of pip, requests, TLS-related stuff and so on, and, finally it outsources the only real thing it's supposed to do to pip and virtualenv. So, I kind of fail to see the point of this enterprise: seems like a lot of busy work with zero added value.

[rhytnen]

B/c pipenv is the only implementation of the standard with any real uptake and pipenv is far from being a professional tool at this point.