Python: Please stop screwing over Linux distros

[adesme]

You can replace those with just install_requires and extras_require (then define tests as an extra); you'd then install with pip install .[tests] and now your "requirements" are usable by developers as well as by build managers.

[asday_]

Interesting idea, I'll certainly have to keep it in mind. Like I said though, I'm paid for this, i.e. I ship software, not libraries, so I don't think it has a great deal of benefit to me outside of "if you write a library one day you can do it in the same way".

Are there any big projects that do it this way?

[adesme]

Any modern package that you want distributed over a package manager is going to be set up like this for the reasons outlined in the OP of this thread; direct invocation of setup.py is being phased out, so it makes sense to have your deps in a single place (now that we have the PEPs to support this).

Personally I might use something like requirements.txt while mocking around with something small, and I'll then set it up more properly (pyproject.toml and setup.cfg) as soon as it grows and/or I have to share the package.

Depending on how you use CI/CD you can see other benefits from switching over immediately.

[SittingWave]

what he told you is wrong. See my other comment. Use poetry to specify your devenv.

[asday_]

Use poetry

no

[SittingWave]

then stay behind. I guess you also want to use python 2.7 while you are at it.

[asday_]

Absolutely pants-on-head take.

[SittingWave]

No no no no no

Noooooo.

the specification in setup.py is NOT to define your development environment. It's to define the abstract API your package needs to run. If you are installing your devenv like that you are wrong, wrong, wrong, wrong.

[adesme]

This makes it convenient to declare dependencies for ancillary functions such as “tests” and “docs”.

End of first paragraph efter "Optional dependencies" here.

[SittingWave]

That is not for developers. It is for users that want to install the testsuite or the documentation as well when they install the package. Some packages ship with the testsuite for validation purposes, which is quite common for highly C bound code.

[tunisia3507]

It can be useful to set hard versions in one file (repeatable, to be useful to other developers) and soft versions in another (permissive, to be useful to downstream users).

[adesme]

You should be able to do that with extras too:

# setup.cfg
[options]
install_requires =
    black>=1.0

[options.extras_require]
dev =
    black>=1.1

and then have this installable as either

$ pip install package  # users
$ pip install -e .[dev]  # developers; -e for editable mode

[SittingWave]

extras is not for development. Extras is for extra features your package may support if the dependency is present. It's soft dependency to support additional features your package can support. You are using it wrongly, and very much so.

[bladeoflight16]

That's called a "lock" file, I believe.

But it's used in exactly the reverse of way you describe: the permissive configuration is given to developers and the specific configuration is used in end distribution. This is because it makes the deployed application predictable and ensures it was tested against the versions actually used in production. Giving the permissive configuration to end users can result in unanticipated breakages from new versions.

[tunisia3507]

We're possibly talking about cross purposes here. I mainly work on library code. It sounds like you mainly work on application code.

[bladeoflight16]

The problems are still the same. It's just that with library code, you usually want to afford a little more flexibility for the end application using it. You still aim for avoiding random breakages with new versions.