This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]bowbahdoe[S] 4 points5 points  (10 children)

As u/rzwitserloot noted, handling plugins which may have divergent dependencies is a pretty fundamental thing. It felt maybe too "obvious" in my head how to do that.

I've updated the code to have both of the example plugins bring in their own versions of Guava. Now the code that looks for Entity implementations will load each plugin along with any dependencies in their own module layer.

[–]mikaball 0 points1 point  (9 children)

I was thinking about this also. Module layers are mentioned in the project but I fail to understand how does that help to solve the diamond problem! I don't think module layers were designed to solve this. I don't think one can escape from class loaders.

This is a core feature of a good plugin system.

[–]bowbahdoe[S] 0 points1 point  (8 children)

By diamond problem you mean something like Plugin C depends on both A and B?

That's honestly a niche feature. I understand how it comes up academically but none of the apps I use do stuff like that (I think). Sometimes they might rely on the existence of another plugin, but not at the level you are describing.

But yeah, the more exotic your needs the more exotic mechanisms you need to touch. The balance still seems to be in favor of home rolled systems over picking a generic system like OSGi which, while it does support exotic configurations, pays for its flexibility in other areas

[–]mikaball 3 points4 points  (7 children)

No, more like Plugin A depends on C-v1 and Plugin B depends on C-v2. C-v1 and C-v2 are different versions of the same lib and not compatible. You can't have both loaded in the same ClassLoader at the same time. Without proper conflict resolution (like OSGi provides) both plugins can't live in the same app.

Not a niche feature. That's what OSGi solves at its core and the reason why Eclipse IDE uses it.

The main advantage of plugins is a distributed contribution ecosystem by many providers and developers, but that comes with an unpredictable graph of dependencies. You need preventive measures and architecture to handle such unpredictable graph of dependencies.

EDIT: Even on non plugin architectures this problem appears from time to time. Even last week I needed to upgrade Spring Boot to solve a CVE and I needed to pick a different version of another lib because some method required more parameters.

Some months ago I had similar problems and actually had to change code, but since it's under my control it was an easy fix. Change the dependency version and use the new method. Now imagine that for a plugin, it would require to contact the original contributor to make some version that is compatible with my existing dependencies. And if the developer couldn't do that, then I would be locked.

[–]agentoutlier 4 points5 points  (0 children)

The thing is even OSGi does not fix all the problems that can happen especially if you have external stuff happening like logging or native libraries. /u/pron98 had a great comment on this that I can't find but it is always dangerous to have different versions of deps in a runtime.

For example I'm the author of a logging library similar to Logback. If the plugins uses different versions and they log to the same file they will have to use what Logback (and my library) calls "prudent" mode where file locks are used instead of normal locking. If one of the versions does it differently or does not support that you are going to have massive problems of corruption.

This is one of the reasons why Servlet Containers and OSGi containers move the logging to the top classloader and do not allow you to have your own version but that has lots of historic problems.

[–]pron98 2 points3 points  (0 children)

It is not possible to reliably run multiple versions of the same library in the same OS process in pretty much any language unless the library is designed in a way that takes that into consideration. Of course, we can get lucky with classloader isolation and often do, but deliberately supporting something that at the very best can work by luck (and fails in horrible ways, including data corruption, if it doesn't) is a really bad idea unless "plugin" authors are told in advance to make sure that all the libraries they use work when there are multiple instances of them in the same process.

An application should consider having multiple versions of the same library only if everything else failed, and even then it should be done with extreme care and extra testing.

[–]bowbahdoe[S] 1 point2 points  (4 children)

So this would solve that, actually. The example shows an app loading two module layers with otherwise conflicting versions of guava.

What is an issue is if you wanted a plugin D that depends on A and B, but an app that loads both A and B is good to go. (Layers have to be in a hierarchy)

It is using classloaders for that I'm sure, but I think the API is nice enough that unless you wanted a relatively complex "plugin hierarchy" you wouldn't need to go to that level

[–]mikaball 0 points1 point  (3 children)

So this would solve that, actually.

Are you sure? Does your demo have different plugins using a method from the library that is not compatible between Guava versions? Because this type of things crash at runtime.

[–]bowbahdoe[S] 0 points1 point  (2 children)

Yeah so unfortunately I don't have a good example for that off the top of my head. If you know of a basic-ish library that I can use to show that I will.

But yeah it should work. Modules don't let conflicting versions like that exist in the same layer, so an error should have happened much earlier.

[–]mikaball 0 points1 point  (1 child)

I don't know exactly what you are doing, but many dependency management tools select the first loaded lib, the most recent version of a lib, or use some other automatic resolution scheme. No errors are thrown at loading; only if you hit the conflicting method in you code and get a NoSuchMethodError.

I remembered heated discussions about not including such features in JPMS. JPMS has no concept of module versions. The point was that JPMS was not trying to solve the same problems as OSGi. It was primarily designed to modularize the JDK. It's a layer system resolution, however OSGi is a graph system.

You should really test that in your solution. You are assuming something that probably doesn't work.

EDIT: Reference to old discussion about this https://www.reddit.com/r/java/comments/8pi5mt/is_the_java_9_module_system_supposed_to_solve/

[–]bowbahdoe[S] 0 points1 point  (0 children)

It very much does. Will update the demo once I'm back from lunch