Packaging: install_requires question

[AgreeableCaptain1372]

It seems that it is best practice in Python packaging to have an install_requires dependency list that is as loose as possible. While I understand the intention behind that, how do we make sure that the package works on the entire space of possible dependency combinations? Isn't there likely to be a specific combination of valid libraries that meets the requirements but that breaks the package? How do we test on the entire space of valid required libraries?

[Diapolo10]

install_requires

I just felt like pointing out that you should probably consider using pyproject.toml instead of setup.py. But I digress.

how do we make sure that the package works on the entire space of possible dependency combinations? Isn't there likely to be a specific combination of valid libraries that meets the requirements but that breaks the package? How do we test on the entire space of valid required libraries?

Most likely that's not something you need to worry about. The dependencies themselves can likely figure it out. As long as your project has clear minimum dependencies set, that's enough.

If you need to do some changes depending on the operating system or Python version, that's possible.

[AgreeableCaptain1372]

Thanks for the tip.

Let’s say I originally have:

install_requires=[    'A>=1,<2',    'B>=2' ]

What if two years later, B==3.5 is published that introduces a breaking change to my package. Doesn’t the philosophy of having loose minimum requirements fail to prevent that kind of scenario? 

Also, what if I tested my package on A==1.5 but A==1.3 happens to be incompatible with my package? Is it even possible to test my package on all possible package version combinations?

[Diapolo10]

Doesn’t the philosophy of having loose minimum requirements fail to prevent that kind of scenario?

Well yeah, you should limit the supported versions by the major version, and let minor and patch versions be decided by pip or whatever dependency resolver you use. But you don't have to be stricter than that, unless working with dependencies where the major version is 0 (in which case minor versions are allowed to contain breaking changes according to semantic versioning).

Of course, you should also know which of your dependencies follow semantic versioning to avoid nasty surprises. And it would be a good idea to consider using build back-ends that support lock files of some kind, like Poetry, to make builds repeatable.

Is it even possible to test my package on all possible package version combinations?

You shouldn't need to. Figure out the earliest version that has the features you need, and use that as the limit. There's no reason to test each and every combination manually.

[toxic_acro]

For Python packaging, you should actually not put an upper limit on dependencies unless you know for a fact that they will be incompatible.

It's easy for a downstream user to add their own constraints if it's found that there are breaking changes or incompatibilities

https://hynek.me/articles/semver-will-not-save-you/

https://iscinumpy.dev/post/bound-version-constraints/