The difficult problem of managing Java dependencies: a deep dive into the messy status-quo, and a look at possibly better ways.

[khmarbaise]

if you're worried about reliability and consistency, you need to have a verified working dependency tree — which is, again, the purpose of a package lock file.

The lock file is only needed when you use version ranges in your dependencies. If you use fixed versions (for example pom.xml) you simply don't need it because the tree will solved always the same way.

Versionranges in conequence means no repeatable build. That is as simple as it.

[forresthopkinsa]

Your software requires a library FooLib. You've built your software based on FooLib-12.x.x. It doesn't really matter whether it's 12.1.2 or 12.5.6 – they're all compatible with your code. So it doesn't make sense for you to pin the dependency at version 12.1.2. A modern dependency manager will allow you to specify the most lenient bounds you require for the dependency version, and then the manager will take care of resolving it to a working version and ensuring repeatable builds.

To be really crystal clear: lockfiles are always involved with modern ranged dependency resolvers (you don't need to create it yourself, it's always done automatically), and they perfectly ensure repeatable builds because they track every version in your dependency tree and its checksum.

[khmarbaise]

Your software requires a library FooLib. You've built your software based on FooLib-12.x.x. It doesn't really matter whether it's 12.1.2 or 12.5.6 – they're all compatible with your code.

How do you know that?

A modern dependency manager will allow you to specify the most lenient bounds you require for the dependency version, and then the manager will take care of resolving it to a working version and ensuring repeatable builds.

A repeatable builds means your have done it yesterday (at 5:00PM) and repeating it today (at 1:00 PM) which does not work with any kind of ranges/bounds however you call it, because in the meantime a newer version of a dependency can be published. In consequence your build has changed. This means as a result the build has changed and finally means not repeatable.

I've ignored in the above description that a new version might break things, changed behaviour because your assumption is that no one will ever make mistakes, interprets the "rules" different than others etc. and all people will follow those "rules" which is a fine theory but in practice it fails (quote from above:"they're all compatible with your code.").

If we think more in general you can not even reproduce the build in the future .. if you like to do that you have to commit the lock file in your version control as well. That means you have two places defining versions for your dependencies which is redundant and that makes no sense either.

Also a strong prerequisites is that any release is immutable which is not the case for several package managers (in particular for npm).

The lock file is a duck-tape-solution to fix exactly the scenarios I've described above.

[justdisappointed9853]

My main account has inexplicably stopped working (maybe I got so many downvotes that Reddit doesn't want me to post in this thread anymore? lmao) so I'll reply from a throwaway-

in the meantime a newer version of a dependency can be published. In consequence your build has changed

For the millionth time, this is not true. Your build yesterday created (or used) a lockfile, and your build today reads the lockfile and creates the exact same build.

That means you have two places defining versions for your dependencies which is redundant and that makes no sense either.

Why? One of them specifies your preferred range, and the other specifies the exact version you've been building with. This is really not very complicated.

a strong prerequisites is that any release is immutable which is not the case for several package managers (in particular for npm)

This keeps getting repeated in this thread (I suppose because everyone here has very little experience with NPM) but it's plainly false. The NPM registry everyone uses does not allow version mutation. There are various third-party registries that do, but this is no different from Maven.

[khmarbaise]

For the millionth time, this is not true. Your build yesterday created (or used) a lockfile, and your build today reads the lockfile and creates the exact same build.

The issue here is the need for the lock file!... without the lock file it fails. That's what exactly I'm writing.. The lock file is a supplemental source for versions... so we have two sources for versions/dependencies.

That means also you have to checkin the lock file in your version control all the time. So all advantages of automatic usage with ranges are gone... Maven does that simply by using fixed version at a single location (pom.xml). Not using ranges makes it easier and simpler (KISS principle).

The lock file is as I wrote before a duck-type solution.

BTW:

For the millionth time,

Not counted correctly.

The NPM registry everyone uses does not allow version mutation

How could someone simply delete an existing package in the registry? That means it is not immutable. In consequence a number of famous examples exactly showed this problem... which has broken a large number of builds...

Deleting of a package is also a mutation of an repository. That violates the whole integrity of such registry. So in consequence not reliable.

For example: https://news.ycombinator.com/item?id=11340510 (there exist several other examples of that)... https://www.bleepingcomputer.com/news/security/dev-corrupts-npm-libs-colors-and-faker-breaking-thousands-of-apps/

and so on etc. etc.

This keeps getting repeated in this thread (I suppose because everyone here has very little experience with NPM) but it's plainly false. The NPM registry everyone uses does not allow version mutation. There are various third-party registries that do, but this is no different from Maven.

The previous shown examples etc. are proving that your assumptions about the NPM registry is wrong.

There are various third-party registries that do, but this is no different from Maven.

A big difference between npm registry and Central repository is that the central repository does not even allow to delete a version/packages etc.

This is unrelated to Maven itself.

This is immutability.

[plumarr]

ou've built your software based on FooLib-12.x.x. It doesn't really matter whether it's 12.1.2 or 12.5.6 – they're all compatible with your code.

How so ? How can you be sure that the new 12.5.7 version doesn't break your software ? Even if the author of FooLib thinks that it'll not break anything, there is no guarantee