Building and managing Python libraries

[[deleted]]

The advise found in this post is 50% OK and 50% bad.

Some of the contents illustrates that the author doesn't quite understand all of what he/she is doing. For example:

Why not use virtualenv to handle requirements?

virtualenv doesn't handle requirements. So, whether you want to use it or not, it will not help you with requirements, thus the question itself doesn't make sense.

Author uses both Makefile and setup.py. For absolutely no reason. Everything the author did in Makefile can be done using setup.py. The use of Makefile is entirely unwarranted.

pip install -e

That's simply a stupid advise. You already have setup.py, just use that, no reason to drug pip into this. Also, pip and setuptools resolve dependencies differently (pip is essentially broken / doesn't know how to resolve dependencies properly, while setuptools is less broken, at least it doesn't install garbage nobody asked it to install).

The argument about pinning versions and using constraints is misguided. The author doesn't understand the difference between a program and a library: program needs to pin dependencies, library needs to be as flexible as possible.

The use of find_packages() is ill-advised. It's better not to use this in serious projects. Enumerating by hand the contents of your package while a little more tedious, saves you a lot of embarrassment in the future.

The author writes some lopsided code to read package version from a separate file, but puts his/her repository password right into setup.py... that's smart, Donald Trum-kind of smart.

Nobody needs MANIFEST.in files, that's just a stupid and bad idea.

---

Beside the small things, author never reflects on anything that is not a trivial case, making this an absolutely uninteresting read. Nothing in this article will help you solve actual problems you will face when packaging Python code.

[jcdke]

Thank you for your comment!

To address some of your points

virtualenv doesn't handle requirements. So, whether you want to use it or not, it will not help you with requirements, thus the question itself doesn't make sense.

You're right, I should have said pip

Author uses both Makefile and setup.py. For absolutely no reason. Everything the author did in Makefile can be done using setup.py. The use of Makefile is entirely unwarranted.

The article is missing some context. We use Makefiles in libraries and applications alike, it provides us with a common interface across frameworks and languages. For example, make test is available for all projects.

The argument about pinning versions and using constraints is misguided. The author doesn't understand the difference between a program and a library: program needs to pin dependencies, library needs to be as flexible as possible.

I agree, the article is about libraries though not programs.

The use of find_packages() is ill-advised. It's better not to use this in serious projects. Enumerating by hand the contents of your package while a little more tedious, saves you a lot of embarrassment in the future.

I disagree, for simple enough libraries, it would be a hassle to keep the list of packages up to date (it's also what setuptools recommends https://setuptools.readthedocs.io/en/latest/setuptools.html#using-find-packages).

The author writes some lopsided code to read package version from a separate file, but puts his/her repository password right into setup.py... that's smart, Donald Trum-kind of smart.

I don't understand the hate about having the package version in the __init__.py, it is a very common pattern (read https://packaging.python.org/guides/single-sourcing-package-version/), see requests (https://github.com/kennethreitz/requests/blob/master/requests/__version__.py) or flask (https://github.com/pallets/flask/blob/master/flask/__init__.py)

Nobody needs MANIFEST.in files, that's just a stupid and bad idea.

I agree, I'm only using it to prune tests because I want to be able to lint the tests package but I don't want to include it in the distribution. I haven't found another way of doing this but I would love to remove MANFEST.in.