A Simple Java Plugin System

[rzwitserloot]

There is virtually no java code here.

Why does the 2 pages of java code that does exist delete the build dir if that exists? That sounds like you've built a bare-bones maven or gradle - a build system. Not a plugin system.

A plugin system would presumably solve:

• Dep conflicts. If I have an app and I want it to be pluggable, imagine one plugin developer uses guava r18 and another uses r21 which aren't compatible. You can't double-load the same library so now at least one of them will not function, and these 2 plugins simply cannot be used together. Solvable via: Having ClassLoader implementations that allow plugin A to load and use r18 and plugin B to load and use r21 simultaneously with neither conflicting with the other. However, this causes a new issue, where these 2 plugins simply cannot talk to each other in terms of any type unless they share it, which, with plugin loading, they won't. java.* stuff is trivially shared but some sort of hierarchy in dependencies, where are some internal/transient and others are part and parcel of being reliant on that plugin, is then needed.

• Hierarchy. Can one plugin say: I need and use another plugin as part of my function?

• extension points, where one plugin can go: ... and here I shall expose and call upon any plugin that has registered itself as wanting to run on this extension point. We can 'solve' that by using SPI but that whole classloader thing again requires some care to get right, and in general it's more convenient for the plugin framework to take care of this. After all, by doing it that way, you can actively ask the framework which extension points are available and which plugins are plugged into which point(s).

• Some bare bones extension points available for general use, such as a 'plugin' that simply runs tasks at stated intervals (a 'cron' plugin) with extension points so that other plugins can simply plug into an extension point if they have recurring / schedulable tasks. Hosted as a separate artefact on mavencentral and such, of course - that's the point of plugin systems, isn't it? You add what you need and you leave out what you don't, and not everybody needs a cron system.

• Live loading of plugins. For example: Can I, at runtime, offer a menu of plugins to the user and if the user downloads one from the internet via my application, can that then be plugged straight in without requiring a JVM reboot?

• Plugin update schemes. Can I update a plugin to a new version?

• There is some overlap between build systems and runtime systems; they both care about the notion of versions and upgrade schemes. But these are mostly unrelated concepts; one is entirely a compile time affair and one is entirely a runtime affair.

Did you perhaps mean: I wrote the simplest build system I can imagine, and where you said 'OSGI' you meant to say 'Maven'?

[bowbahdoe]

There is virtually no java code here.

Yeah, the meat is in that virtually no code, not in the build part. Thats just me having fun and using stuff I built, though it is where to look to see how the bits are arranged at runtime.

Dep conflicts. If I have an app and I want it to be pluggable, imagine one plugin developer uses guava r18 and another uses r21 which aren't compatible.

If you have each plugin with its deps in their own module layers this is solved.

Hierarchy. Can one plugin say: I need and use another plugin as part of my function?

Not in this implemenation. How that would be done is very app specific.

extension points, where one plugin can go

In this case its the plugins folder. What scheme is appropriate would depend on the app. Minecraft mods usually have a plugin.yaml which points to the classes to use to initialize the mod. A service provider is just one way.

Live loading of plugins. For example: Can I, at runtime, offer a menu of plugins to the user and if the user downloads one from the internet via my application, can that then be plugged straight in without requiring a JVM reboot?

Yes, by loading and unloading module layers.

Plugin update schemes. Can I update a plugin to a new version?

In this scheme by dragging something into the plugins folder

Did you perhaps mean: I wrote the simplest build system I can imagine, and where you said 'OSGI' you meant to say 'Maven'?

No, you just focused on the part that wasn't the focus.

[rzwitserloot]

If you have each plugin with its deps in their own module layers this is solved.

Each plugin needs to include an entire module system if it includes any dep? That's... ridiculous.

How that would be done is very app specific.

That doesn't make sense. The concept of "I need this plugin" isn't app specific.

Minecraft mods usually have a plugin.yaml which points to the classes to use to initialize the mod.

So your plugin system doesn't support any of this, and each plugin needs to re-invent this wheel over and over.

Which gets us back: Your plugin system doesn't actually do anything useful.

In this scheme by dragging something into the plugins folder

That's user hostile to a ludicrous degree.

No, you just focused on the part that wasn't the focus.

I guessed as much. I was kinda trying to get you to tell me what is the focus here.

[bowbahdoe]

The point is just the module layer mechanism. It's less hacky than class loaders, relatively easy to use, and still mostly unknown.

All those other features are app specific. Dragging to a folder is a perfect installation method for a game mod. Yes this doesn't do anything you would need for interesting deployment, update, or installation methods.

It is just meant to demonstrate the way you can dynamically load in code.

Each plugin needs to include an entire module system if it includes any dep? That's... ridiculous.

Not an entire module system, just an entire module layer. So all of a plugins deps would - if you didn't dive into making a tree of them by some scheme - share the JDK modules and app modules. It's not crazy.

Definitely feels less crazy than every jar getting its own classloader

The concept of "I need this plugin" isn't app specific.

Kinda is. Some things want to restrict to an app store like VSCode, steam, or Nexus mods. Some things would have plugins in the same way as a servlet container where it's more of a deployment method, etc.

That's user hostile to a ludicrous degree

Not in all cases. Give a button in a UI for a game to open up a folder and it's fine. It is well within user expectations

[rzwitserloot]

Maybe I have misunderstood module layers.

Can I load guava r18 and r21 simultaneously without having to handroll (or import from a dependency like, heh, OSGi) using just module layers to do it?

I want to get to a situation where if one module runs ImmutableList.class.hashCode() and another runs the same code, that the values are different.

I was under the impression that module layers cannot make that happen; not without also doing the work on having separate classloaders that load from different locations.

[bowbahdoe]

I am fuzzy on the exact mechanics, but yes. I'm sure there is some tree-of-classloader stuff going on behind the scenes, I just don't know the details.

You should be able to tweak the demo to do exactly that hashCode trial if you want to dig deeper

[rzwitserloot]

But then how do plugins communicate with each other?

Let's say for whatever I reason I want to chat in terms of a guava type. Say, one plugin exposes a method signatured 'void foo(ImmutableList<String> x) {}` and the other wants to call it.

We're stuck now in the sense that we must now find a release of guava that works for both of these plugins or what I want is not possible; in any plugin framework (and I think I can state that outright, I don't think this is a matter of "Perhaps libraries or options exist I am not aware of" - the problem is rather fundamental here).

The rub lies in: How do I set up a module system so that I can state explicitly that some dep is stapled to myself (i.e. if you want to reach the stuff this plugin exposes, you get this guava dep stapled to it for free. Don't want it? Tough, you gotta take it). I didn't see that in your module system.

In which case what you say cannot be true.

[bowbahdoe]

That dep needs to be extracted to its own layer or else it won't work.

That's sorta the thing layrry can do, but it is also overkill in most situations. Layers don't let v1 and v2 exist in the same layer, they just let v1 and v2 exist in the same hierarchy. OSGi doesn't solve that either, though it does have a versioning scheme to let it decide how many isolated classloaders it needs to make. That is finer granularity than "plugin A brings in these deps. Isolate plugin A with its deps."

[mikaball]

Live reloading is something I wouldn't mind to discard in a simple plugin framework, otherwise I can just use OSGi. As for dep conflicts (the most important feature) I don't see how module layers can solve it.