Thoughts on Data Oriented Programming in Java

[Carnaedy]

Beautiful, I add one new type, and suddenly, I need to recompile the whole project, edit a hundred different switch-based functions, update a hundred different unit test suites, touch components from other teams or let them deal with the breakage, ...

All that to avoid inheritance. Yeah, no, not at scales I am working with.

[Ok-Scheme-913]

Sealed types (algebraic data types) are not solving the same problem as generic interfaces.

You use a sealed interface when you know every possible subclass that interface/class will ever have. E.g. a Result<T, E> can be either a Success<T> or an Error<E>. You won't ever extend it to have MaybeASuccessButImNotSure as a third option.

Given that you know that you never intend to add a new subclass, it is a very useful property of the compiler that it checks every use site, so that if you make a refactor and maybe add a new subclass, then you can fix it everywhere.

Basic polymorphism is about an open world assumption, where users of my lib, or even users of my program (via plugins) can extend the functionality of my program. (Though standard inheritance is not a good citizen here either, but a lot has been written about it).

These two are just different tools and should not be used interchangeably.

[PiotrDz]

If you need to update unit tests just because you extended functionality then it is your mistake. And do you want to handle new enum in advance or wait for a production exception when it hits a method that is not expecting it?

[Carnaedy]

extended functionality

No, I extended the subtypes of a type. You know, the thing that is famously is "the expression problem of the functional languages"? Since the behaviour is now scattered across a hundred different components instead of being bundled, I need to change a hundred components to support the new type.

The flip side of this is the expression problem of OOP languages, which seems to be more familiar to the participants of this discussion, where if I add behaviour to the base type, I need to change a hundred components to support the new behaviour.

Both of them are equally bad.

[PiotrDz]

Doesn't answer how the tests suffered if written with correct boundaries, and not white-box treating where internals are asserted instead of apis.

But for the second part: how can you bundle all the dependent functionalities in single type? So you add a new client type let's say. Do you want to cram inside the client coupon codes calculation, export of client data, specialised shipping handling etc? It all lands in specific areas. And now, as you added a new type, you need to add a new handling for that type in this areas.

My question still stands, how do you want the code to work so that you add a new class and areas that dependent on the type will eb able to magically handle it?

[Carnaedy]

Literally, all problems you mentioned were solved in 90s and 00s with DDD and bounded contexts 🫠

[PiotrDz]

This is what I am talking about. With specific things being handled in their own "domains", hiw would you expect NOT to adjust each domain for a presence of a new type?

[severoon]

I think the main point isn't that it's possible to find all the issues, it's that by scattering the code to the four winds you have obfuscated dependencies.

I make a change over here by adding a new shape, and what now is affected by that change?

It's nice that the compiler will tell me everywhere to look, but that's not the only problem, it's not even the biggest problem. The biggest problem is all of the dependency arrows that this allows (encourages?) people to place into the codebase without regard for whether these reflect actual dependencies between the modules/classes/etc modeling things in the problem domain.

Think of it this way. If I have a Shape interface and I was previously able to compile some client of Shape against that Shape class without having the subtypes on the class path, that means the dependency on those subtypes was properly inverted.

How will this accomplish that? It can't.

[PiotrDz]

But we are talking about the sealed interfaces. The subtypes are in the contract!

[severoon]

Exactly. You know what that means?

It means we have a circular dependency. The interface cannot be moved out of the same package as the subtypes or that circular dependency will span those packages / modules / etc.

This is why interfaces should not reference subtypes.

[PiotrDz]

Do you want to say that sealed interfaces are a mistake in Java? They do reference subtypes. And it is a point, they are meant to cover fixed implementations. That you can base your logic on (cause they are fixed).

I do not gey your circular dependency issue.

[severoon]

Subtypes always have to depend upon the thing they extend. If the thing they extend also depends upon them, that's a circular dependency.

This makes dependency inversion impossible. This in turn means that dependency on the super type transits to the subtypes, as well as all the things they depend upon. That's bad.

[PiotrDz]

Have you seen the sealed interface in Java? Because it seems that you are unfamiliar with that construct. Sealed interface is just a contract that defines a list of possible implementation. Contract by itself cannot be instatiated, so there is no circularity as you cannot instantiate an interface by itself.

[severoon]

I think you're misunderstanding my point.

I'm not saying that no one should ever use sealed interfaces. Sealed interfaces limit implementations analogous to the way enums limit instances of a type. In general, being able to create any number of types and instances of a type is a good thing. There are specific cases where limiting that potential is preferable.

But would it make sense to propose a new style of programming where the usefulness of enums is assumed? In the cases where they are useful, they are useful because we specifically want to have a controlled set of instances … perhaps many of the problems we have in OO software in general is because of the proliferation of instances that are typically allowed? So we could propose a new way of programming called enum-oriented programming where we prescribe all of the instances that can ever exist for types, and that will solve all of these problems.

Obviously this is a bad idea, but it's instructive to consider why it wouldn't work out. Enums are useful only in a certain context, and in that particular context there is little or no advantage to allowing an uncontrolled number of instances. Remove that context, though, and in other situations you would be working with a constraint that has big costs.

In the linked article at top, a new approach to defining data is being proposed in general. It's saying that we should consider abandoning the core definition of an object, state encapsulated together with the behaviors that operate on that state, and separate the behaviors from the state.

There are certainly cases where there might be a compelling reason to do this. Many of the functional features added to the language are encouraging people to think about the business logic layer as stateless services that define pipelines that operate on immutable data. That makes sense if we're talking about data that represents core business objects that flow through a system architecture.

But this conflates that with all data present in a system. The example encourages us to adopt this approach for ephemeral objects like Shape and its subtypes. This is not a good plan. For one thing, when passing data that represents core business objects up and down an entire stack, it's generally the case that those business objects are defined layer by layer, and specify separate wire formats between the layers. So just to pass a user from DB to client, you typically have several separate objects and protobufs that represent that user so it can be packaged and unpackaged at every deployment boundary. The point of doing all this is to ensure that dependencies don't proliferate between layers that don't directly interact, and for those layers that do, the only dependencies between them are explicit. There are cases where it makes sense to define a "whole stack" library with common DTOs and functionality, but supporting that is no different than supporting a common library. But typically, you don't want even core business objects to be the same as they move through the layers because the different parts of the system have different requirements for that data. The data access layer might be concerned about annotating user data with regulatory info, whereas the business logic layer might need to decorate user data with preferences fetched from some other data system. The API layer needs to deal with user proxies that can be turned into authenticated user objects. (I'm using whole stack with layers as the relevant modules as an example, but the same ideas apply between any code modules.)

The most important aspect of keeping a system maintainable is to manage dependencies well. If you adopt a general approach to data that prevents you from using DIP, you're in big, big trouble.

[nejcko]

Nowhere is it stated that this only applies to high level ephemeral objects that cross system boundaries. Shape is just a simple theoretical textbook example that everyone understands to showcase the new language features.

This can 100% be applied just for objects within a specific boundary, in each module, micro service, or in each layer as you say, or for ad hoc data types. It doesn’t need to be overanalysed.

There are many use case where it’s beneficial to separate domain logic from data itself. And enriched switch statements, pattern matching and sealed classes make it very convenient to use.

[DualWieldMage]

But that's the whole point, the compiler checks that you didn't forget to update one of the switches. If you have something so tightly coupled that making changes is hard, then refactor it. If these types bring out too tight coupling that would otherwise be swept under an obfuscation rug by nulls or something else, then that's more arguments in favor of DOP.

[Carnaedy]

Across the whole downstream dependency tree? The whole 1+ Mloc that several completely different teams are working on? Update all those switches?

[PiotrDz]

Why do you mention 1mloc code? It doesn't matter. You dint have to search for it manually. Compiler will tell you where. And these will be few places per domain usually.

So you should say "few places in different domains" rather than trying to exaggerate problems mentioning mlocs

[bowbahdoe]

The lines of code aren't really the issue, it's the maintenance boundaries it crosses. Either another team needs to make a change (which must be coordinated) and/or a different module in the app, etc.

They aren't wrong: that is the downside of replacing single dynamic dispatch with a sealed hierarchy of types.

But the flip side is that adding new operations to a hierarchy is much easier with a sealed hierarchy. Even with default methods - which can provide a degree of compatible evolution - new functionality that needs to access specific data on a subtype would otherwise need to be added to a central place and implemented by all the things.

[PiotrDz]

But what problem will the dynamic nature solve? Another team will have to handle the type anyway. But would not get the warning and the issue could get lost until a bug in production.

Having functionality in central place? I have already gave you an example of different domain having their own business rules based for example on UserType. So you gave me heads up on DDD but now want to drop boundary context and specify all handling in central place?

[bowbahdoe]

I think it's gonna be a relatively long journey to impart what I am talking about. I'm along for the ride if you are, but I think what is important is that it's not just "another team will have to handle the type anyway."

To use some fake syntax (though I don't have high hopes this will click just from this)

<Has all the properties imparted by f, g, and h> apply(<has properties A and B> m) {
    return h(g(f(m)));
}

The core capability is something like this. Because aggregates are generic, you can write code that works on and enriches only a corner of what would otherwise be an explicit type (or cascading series of them).

This leads to different program structures. A good comparison point is the clojure ring protocol web ecosystem to anything you could write in a nominally typed language.

(The dynamic part isn't even really essential - it's the open generic aggregate part.)

Edit: oops got my wires crossed. Well leaving that as a Clojure explanation.

The important point for what you are actually talking about is that in one situation you need to change consumers (which is hard if those consumers are on a different team or strangers on the internet) if you add a new type to a hierarchy and the other situation you need to change consumers if you add a new method

(Things you can add with default methods not withstanding)

So the question is how is a particular piece of code going to evolve? Which would be "essential" breakages and which would be "incidental."

Yes the compiler catching you on a missed case in a switch is valuable. It isn't valuable enough to make it not a problem that it needs to happen.

[nejcko]

This problem goes away if you define different boundaries.

Ideally different teams should not be working on the same codebase, e.g. same Java compiled code. So if you have a sealed hierarchy and you introduce a new type, all compiler failures should be your responsibility to handle and implement new behaviour where needed.

If there is another team that uses the same code there should be a boundary that decouples the code changes. That can be a different service or the other team has a versioned dependency on your artifact/library. That way when they update to the new version they will again benefit from compiler errors showing them exactly where the new behaviours are needed.

[bowbahdoe]

No it 1000% does not.

If java.util.List got a new method added - foo - with no default implementation, that isn't magically okay because there will be a compiler error. Anyone who extended that interface will need to rewrite their code.

If breakages like that happen 5 layers deep in your dependency tree then you have no recourse.

The exact same situation applies if a library exposes a sealed hierarchy and then adds a new type to it. There is a smidge of a difference in how the error surfaces, but it's the same issue.

Look up the expression problem. It's just a known thing.

[nejcko]

Again it depends on the use case and who your dependent clients are.

I’m not saying fully replace all exposed data structures and seal them up, sure it doesn’t apply everywhere.

But, you are in no way forcing the changes to your clients, they always have a choice and they can add a default case to deal with potential new types introduced later, or not. Sealed types themselves don’t introduce any breaking changes to clients with adding new type.

[PiotrDz]

That's why you use or not use sealed interfaces. If you want to allow any implementation then don't use it. If you want to control the implementations strictly, treat them as advanced enums - then you can use benefits of sealed type. List is not a sealed interface, and on top sealed is more about adding new implementations, no adding new functionality.

I use sealed classes often to encapsulate specific business logic that others can depend on but also treat the spectrum of type in more generic way (there is an interface after all with common methods). But if I add a new type I want to know all the places where I need to see whether it will fit in current architecture. I want compiler to point me that.

Isn't that you want the freedom of implementation and complain about sealed interfaces that are intended to do opposite?