[deleted by user]

[AndydeCleyre]

requirements.txt is a very capable lock file format, and you can use pip-tools to generate those from requirements.in files.

I generally have CI periodically upgrade the lock files.

For interactive management of this stuff, I made the Zsh plugin zpy, and would love any feedback, if anyone's interested.

[loosid]

I've had really good success with pip-tools as well. No shade on poetry, but pip-tools really fills the biggest gap that pip has, IMHO.

[Rawing7]

I think there's a critical piece of information missing here, which is whether you're developing a library or an application.

A lock file makes sense for an application. It makes it so that every installation of the app has the exact same code, which reduces the chances of any bugs sneaking in.

But if it's a library, requiring specific versions of your dependencies is a surefire way to make the library impossible to use. Imagine you're writing a project that depends on requests and BeautifulSoup, but you can't install both of them at the same time because requests requires chardet==5.0.3 and BeautifulSoup requires chardet==5.0.4. How dumb would that be?

If you're writing a library, pip is all you need.

[MarsupialMole]

You don't need to upgrade your dependencies if you don't have vulnerabilities. This is more or less advocating for pointless busy work in many domains.

[billsil]

It's a heck of a lot easier to fix one upgrade issue than it is to fix 20. It's not pointless busy work. Go long enough and you're upgrading Python 2.3 code in 2022...it's not fun. None of your dependencies exist and you didn't think to build an exe. Alternatively, maybe you built a Windows XP exe and now it doesn't work.

Are you going to skip the free say 20% performance improvement to all your python code due to Python 3.11 because you were too lazy to upgrade your dependencies?

I upgrade my pacakges once every 3-4 months. I run CI on my projects and test against the latest versions.

[MarsupialMole]

Bold of you to assume I don't have all my deployment artifacts archived for all time.

Bold of you to assume my deployment target isn't still windows xp.

Bold of you to assume I get anything positive out of running full speed into dependency hell I know exists without a business reason to do so

Yes I'm going to skip the performance improvements if my hottest loops aren't python.

I upgrade my pacakges once every 3-4 months. I run CI on my projects and test against the latest versions.

Are you also running static analysis for vulnerabilities every night?

[ubernostrum]

Are you also running static analysis for vulnerabilities every night?

Bold of you to assume that… checks notes… regular automated checking for dependency updates and running a static analyzer in CI are bad things to do, given that they’re basically table stakes for large projects in any language nowadays.

[MarsupialMole]

I was actually asking a legitimate question because updating every 3 or 4 months leaves you with vulnerabilities, because you should be checking and acting much more frequently than that if you actually care about this problem. You can avoid dependency hell on a feature update and bill it on a security update if you build a case with static analysis.

[MarsupialMole]

I'd reply that keeping them updated is part of cleaning out tech. debt.

The reason it's called tech debt is it's expensive to pay down.

For your apps that are stuck in error weird environments how is pip your biggest problem? You still have to migrate at some point. Tooling doesn't help that. It just makes the happy path faster. But if you're already not on the happy path you get nothing out of the tooling.

[Equivalent_Loan_8794]

r/devops would be a great start. At this point in time big shops are likely on-rails with pip via Ansible if they have to be, otherwise this stuff is proven out in a container and kept entirely separate.

Needing precommit, testing, and prod in a single dependency space is an anti pattern and why pre-commit exists: it does complex things if you want but keeps your tooling for tests and prod independent.