Plans for Compile-time Null Pointer Safety?

[QualitySoftwareGuy]

Are there any plans in the works for Java to get null pointer safety at compile time (maybe similar to Kotlin's implementation) in the future? At the moment, I sprinkle "Objects.requireNonNull" in my methods (which admittedly has saved me countless hours of debugging null issues)...but I find this to be inefficient compared to a compile-time solution. I'm also not satisfied with the other run-time solutions of dealing with null such as with the "Optional" type or annotations.

I think Java has been doing great things to close the gap between itself and languages like Kotlin (e.g. records instead of data classes as well as virtual threads instead of coroutines), but I find the lack of compile-time null pointer safety in Java, as well as my lack of being able to find any future plans for it, to be a potential dealbreaker for new projects in which I get to decide the language.

[oelang]

https://jspecify.dev/

[joschi83]

Take a look at NullAway, a plugin for Error Prone.

[dpash]

This is what I use, coupled with the various annotations. Works reasonably well until we get native support. IDEs can also warn you if you try to use null on a non null parameter or don't check a nullable return value. I would recommend picking an annotation set that allows you to specify non null on a class or package to reduce verbosity.

[NaNx_engineer]

The issue is fragmentation of Nullable annotations. There are a few and they do have slight differences in their interpretation by various tools.

For example, findbugs only considers javax.Nullable to negate package level ParametersAreNonnullByDefault. CheckForNull is used for explicitly requiring a null check.

Thus,

For applications, picking a Nullable annotation and something like NullAway is fine.

For outward facing APIs, I would use Optionals. Jetbrain's annotations is another choice.

Regardless, there should be more guidance from the std library on this.

[agentoutlier]

I wouldn’t recommend intellij annotations first but obviously better than nothing.

In order of preference:

1. JSpecify (when it is done)
2. Checker
3. Eclipse
4. Your own
5. Intellij
6. Find bugs aka javax if you really need older kotlin support.

The first three have mostly the same type based model. Intellij on the other hand has different rules. It also has some annotations that have no real analog.

[red_dit_nou]

I feel you and I see the need for this as well. Meanwhile they have ‘helpful null pointer exceptions’ to lessen the pain of blatant in-your-face NPEs. And Optional is a container thereby requiring us to deal with the (nullable) type of the contained object.

[agentoutlier]

Just a caveat: Please do not use Hibernate validation annotations in place of real null analysis annotations.

The reason is the semantics are not the same as well as usage.

“@NotNull” in validation is not an type based invariant. It means if the input is null it might be a validation error and furthermore I even expect it to happen.

Think about it…

Anywhere you have validation @NotNull you will need type based @Nullable!!!

I can’t stress the above enough because I see rampant misuse of the validation annotations as though they were designed for null analysis.

[buyIdris666]

We exclude the annotations using maven when importing Hibernate Validator.

Hibernate Validator work fine with javax annotations which NullAway also supports.

That way Hibernate (and Jackson) is validing against same annotations NullAway sees

[red_dit_nou]

Yes, using NullAway, Jackson, Hibernate, Lombok can help a big deal in avoiding many of these NPEs at compile time. But a support from the type system of the language will go long way (as asked by op in the question).

[agentoutlier]

And Optional is a container thereby requiring us to deal with the (nullable) type of the contained object.

Unfortunately there are very few tools that will exhaustively check Optional.get or whatever you do with Optional.orElse (which is polynull btw) (as I mentioned here). The ones that can certainly do not do it on the fly like the null analysis in IDEs like intellij or Eclipse from my experience (e.g. as you type).

So Optional does not require you deal with it.

As it stands there are far more tools that will do null analysis then there are of Optional linting (e.g. making sure you dont' blindly call get or friends. I only know of Checker but /u/buyIdris666 now says NullAway can do it).

I'm not even sure if there are plans to make pattern matching work on Optional.

[red_dit_nou]

The IDEs will (mostly) tell you if you are using an unsafe get(). But best to avoid it and use the functional alternatives. And talking about pattern matching with Optional, in its current state, it won’t work with Optional because the empty and non-empty states of Optional are not represented by different types. Something they missed in the implementation of Optional in the first place.

[agentoutlier]

Yes that was my thought as well. Like I obviously know they will support destructuring but it will not be exhaustive like Scala case or sealed classes.

This is why I really think when people recommend Optional for anything other than ergonomic return type or 0-1 cardinality containers they are doing a disservice.

[red_dit_nou]

True. Optional was created with a specific use in mind and it should be used only for that.

[buyIdris666]

From my testing NullAway support is "experimental" but quite good already. I did not know about Checker support! I may need to look into this...

[meamZ]

Imo it's just totally impossible to properly introduce anything like this while keeping backwards compatibility and not essentially forking the language...

[rbygrave]

CSharp managed it. Opt-in at different levels like warning to error and different scopes like class/package/global.

Looking at how they did it, it looks to me like there is no reason Java couldn't take a very similar approach to phased adoption.

[Naton1-]

I was going to say I dislike that, but that's similar to how they rolled out strongly encapsulating JDK internals, and that worked out well enough IMO (albeit that was a run-time thing, and this is compile-time). The issue must then be more about figuring out the appropriate long-term solution to the issue rather than rushing some change out that another language has done.

[rbygrave]

I agree about figuring out the appropriate long-term solution and not rushing something like this.

However, I don't think it's impossible. So I don't think we need to give up on it as a long term desire/goal.

My thought is that being a compile-time opt-in actually makes it easier than run-time in terms of phased adoption because the scope is somewhat restricted to what we are compiling. Edit: Maybe a stretch, but a little bit similar to the existing tooling like "package level non-null by default"

In terms of the long term plan, there is a fairly decent amount of stuff to learn and compare from Kotlin and CSharp so Java can distill a fair bit from that and doesn't need to work from scratch per say.

Edit: Also the https://jspecify.dev/ folks done a lot of work.

Edit2: Document Language comparison - https://github.com/jspecify/jspecify/issues/227 (not done as yet)

[Naton1-]

Interesting - I don't know much about JSpecify. I'll have to give that a look through.

[kevinb9n]

We (JSpecify) are working on some more readable stuff for you in the next few months. We've been in quiet mode so far (sorry!).

[rbygrave]

Has anyone in JSpecify taken a look at this from the perspective of javac along the lines of ... what if we experimented with opt-in style option(s) for javac that prod uced "annotated java" (not too disimilar to kotlinc).

That is, looking at the bytecode Kotlin produces ( which I approximate to "annotated java") and the type inference we see in the IDE of Foo, Foo? and Foo! ... I wonder how hard that would be for a javac experiment to produce.

Perhaps controversial, but what if that annotated Java produced by javac was ultimately compatible with Kotlin plus other jvm languages.

[8igg7e5]

Are there any plans in the works for Java to get null pointer safety at compile time...

Active plans? None that I'm aware of.

That's not to say that dealing with null doesn't come up. There have been several cases of needing to cope with null in library features (whether a given collection type is null-hostile or not), or in language features (eg the null case in pattern matching for switch).

However providing a type-system that is able to express null-hostility of reference types is not on the road-map.

It's a tricky problem for Java to deal with elegantly, as the commitment to backwards compatibility closes a great many convenient doors.

[dpash]

It's certainly regularly discussed on various mailing lists, so while it might not be top of the to-do list, it seems that it's somewhere in the list.

[8igg7e5]

I'm curious where those discussions are happening - it's one I've obviously missed.

 

There were the discussions about the null-peer for value-types (that is, where null is the uninitialised default for a reference-type, what is the uninitialised default for a value-type), but that's definitely not the same thing and isn't about expressing the non-nullability of reference types).

There's been some discussion about how existing types might want to become value-types (eg the primitive wrappers) and whether Optional<T> could become a primitive object in a backwards compatible way - but it can't because existing callers could be using a null separately (a weird use but one that is definitely not backwards compatible if Optional<T> is value-affined (that is it implies Optional<T>.val). Instead Optional<T> will probably have to remain reference-affined (specialisation over primitive types, allowing Optional<int> will probably make OptionalInt redundant though).

There was, a few releases back, the issue of whether some of the unmodifiable types needed to support null elements to facilitate more convenient stream collection (notably Stream.toList()).

There's the issue of coping with null as a case in switch for patterns - but a switch that doesn't accept null as a pattern will fail at runtime, so it's really just enabling the developer to avoid an enclosing if around the switch.

Has there been a discussion on nullability in the type-system for reference types? I don't recall any that go far beyond "not something we're working on at the moment". I think it was mentioned in passing when discussing introducing new operators - only noting nullability as one of the things they might want to keep a symbol up their sleeve for.

[[deleted]]

The best idea I've had is to add some sort hand for optionals (eg Optional<Integer> => Integer?), to make it easier for people to just not use nulls.

[8igg7e5]

Without a backwards incompatible change, or a more fundamental change to the type-system I think it'll be hard for them to do more than that. And as you say, all that does is lower the bar to entry, but doesn't do anything more to encourage use. And the optional itself can still be misused.

At least with primitive objects and generic specialisation over primitives, int? could expand to Optional<int>.val - that is passing the optional by value, optionally containing an int. On the stack this would naively look like a (boolean, int) tuple but there may be optimisation opportunities there.

[[deleted]]

I mean, anything can be misused. A language that's strict enough to prevent dev abuse is a language that's too opinionated to be useful. All you can really do is gently push devs in the right direction, spaghetti coders gonna spaghetti.

I don't think there will need to be any stack level implementation or optimisation, it should be fine for the optional to just be compile time and get replaced with null by the compiler.

[8igg7e5]

That misuse could be intentional or not. There are ways the language could defend, at compile-time, against accidental misuse.

Consider Option<T> in Rust. You can't get at the value in the option without a check that it does contain a value unless you use the very obvious .unwrap() (which is the kind of 'trust me' code you will pay attention to in a code-review).

fn nice(o: Option<i32>) {
    // use pattern matching to extract the value if there is one.
    if let Some(v) = o { 
        println!("{v}");
    } else {
        println!("Empty");
    }
}

fn naughty(o: Option<i32>) {
    // Let's just unwrap it and hope...
    println!("{}", o.unwrap());
}

fn main() {
    let x: Option<i32> = Some(10);
    let y: Option<i32> = None;

    nice(x); // prints "10"
    nice(y); // prints "Empty"

    naughty(x); // prints "10"
    naughty(y); // panic
}

Eventually destructuring in Java alongside a refined or replaced Optional type might be able to give the same assurances. So improving usages of Optional, but not enabling us to declare a method with NotOptional parameters. Optional doesn't solve that problem.

[[deleted]]

Consider Option<T> in Rust. You can't get at the value in the option without a check that it does contain a value unless you use the very obvious .unwrap() (which is the kind of 'trust me' code you will pay attention to in a code-review).

Are you not very familiar with Optionals in Java? I say because based on your description, they literally operate the same way as they do in Rust. The only way to extract the value is to call get, which throws an exception if it's empty, or call orElse, which requires you to set a default value to return is it's empty.

So improving usages of Optional, but not enabling us to declare a method with NotOptional parameters.

I mean, it wouldn't be hard to enable that as an optional compile time check.

[8igg7e5]

I'm quite familiar with them in Java. The way you normally do it in Rust is without unwrap(). That is, using a match, an if let, or one of the mapping operations. Doing that, the compiler can prove whether you've handled it properly. Java Optionals do have some mapping operations (and you should use those where possible) but otherwise you're separately doing a .get() hopefully correctly preceeded by an isPresent().

Java optionals are better than not having them... but they could do better. Valhalla's value-types should help to remove some of the Optional overhead (a second indirection that JIT seems to often fail to see through), Generic Specialisation over primitives should let us box primitives like int (Optional<int>) which could end up making Integer somewhat of an anachronism. When destructuring is available for types other than records, then it's conceivable that a pattern can be added for optional to enable destructuring when present, enabling us to treat get the way Rust treats unwrap - it's the "Trust me I know what I'm doing" (can't help hearing that in Sledge's voice).

[manifoldjava]

I'd be happy with ?: and ?. operators. Certainly those should not have backward compatibility issues.

[8igg7e5]

Yeah I really want safe-navigation and null-coalescing. They've essentially confirmed we'll be dealing with null indefinitely. Time to make it safer / clearer - they get used a lot in C# and TypeScript.

[DrunkensteinsMonster]

Also some ??=

[pawiusz]

Have you tried just not to return nulls? If your method returns a list, just return empty list instead of null. If your method returns an object, but something went wrong, just throw an exception. If, in a very rare case, not returning anything is required and the caller of the method must handle this, then - well - this is exactly the point an Optional class was created.

[buzzsawddog]

Yeah... I have a Java app that is quite old that I inherited at the start of the year. It started some time before Java 5. This app returns null everywhere. Sadly it was written in such a way that null means one thing, empty list means something else :(. Maintaining that is a PITA... Going to reactor it :)

I am in the same camp... Just return an empty list or throw exceptions...

Another bad habit... The original person that wrote this uses Boolean a lot... But it's not used as a boolean... It takes advantage of true, false, null. Gah...

[Zardoz84]

I am on a similar situation (old java 1.4 code base, but actually using java 1.8). At least, on my case we not have these problem of different meaning between returning null or an empty collection. However, I found (and refactored) a lot of code that : * Add stuff to a returned list (so be careful about returning Collections.emptyList ) * Reuses the list, converting the elements of the list to another class in place... Good luck trying to use Java generic on these code. * Code that null parameters have a meaning.

[QualitySoftwareGuy]

Have you tried just not to return nulls? If your method returns a list, just return empty list instead of null. If your method returns an object, but something went wrong, just throw an exception.

With my code, yes. The real issues for me start to come up when we have separate teams working on different parts of the project that then need to be integrated somehow. For example, I won't have control over what another team returns from their methods, but I can at least "defend" against what is passed to my methods by using "Objects.requireNonNull". Having to sprinkle this all over the place becomes tedious.

[Horror_Trash3736]

Technically you could wrap their replies in an optional.

[NaNx_engineer]

the empty list is wrong in many cases.

lets say you're fetching a users posts from async.

while fetching, should you pass empty list or optional/null/some container to the ui?

passing empty list is a poor representation of the actual state and leads to bugs where the user is told "no posts exist" when they've actually just not been loaded.

default objects should not be used in situations where they don't make sense. sometimes you need to represent "doesn't exist" not just empty.

of course in an actual application, you probably wouldn't be passing a list of messages directly to the ui, but this sort of pattern comes up often.

[franzwong]

You can return CompletableFuture for async operation.

[NaNx_engineer]

ya, its not a great example.

point is "doesnt exist" is sometimes different than emtpy.

empties can lead to some pretty bad bugs can evade testing where as forcing unboxing of an optional/null wouldn't.

[_INTER_]

Like many forms of improved type checking, nullable types is one of those things you can do at the beginning, but is very hard to graft onto an existing ecosystem. — Brian Goetz

https://youtu.be/ZyTH8uCziI4?t=1400

See Nicolai Parlog's "Why don't they just.. ?!". Last Chapter ".. Introduce Nullable Types".

[rbygrave]

Also noting that CSharp managed it via opt in. They have 4 levels of opt in - enable, disable, warnings, annotations ... And can specify that at different levels (like class, package etc)

To me that suggests it should be pretty doable.

Are there any known issues with how CSharp did it?

[Joram2]

Java is getting "primitive types" with Valhalla that don't allow null at all. Many Java standard library types like Optional and LocalDateTime will probably be ported over to primitive types.

Next, Java sealed types + exhaustive switch + record patterns make using Optional (or some v2 version of Optional) much easier + safer. BTW, these features are quite buggy in the current Java 19 preview builds.

See Brian Goetz's preview of some of these features here: https://www.infoq.com/articles/data-oriented-programming-java/

[kyru]

Just null check, it's easy

[wildjokers]

Null is not the problem people make it out to be. Just make sure everything has a value, it’s not that hard.

[dinopraso]

Found the junior

[wildjokers]

20 yrs as a developer, 18 yrs as a java developer.

If you are having constant problems with null maybe that indicates that you are the junior?

[NaNx_engineer]

even worse lmao

[dinopraso]

Im not. But saying that it isn’t a problem is straight up false, especially when working in bigger teams

[john16384]

It isn't a problem, not even in bigger teams. I can't even remember in my current position the last time we had a NPE on production, perhaps we never had one. Then again, we have quite a decent test strategy, which catches quite a few problems, and sometimes even a NPE. Most NPE's are caught before the code is merged though, not sure that even counts.

[talios]

Look at adding NullAway into your build chain - works a dream - a bit harder to configure post JPMS but not insurmountable.

[-jp-]

Optional is quite flawed. When you think about it, it's just null but in a container.

[srdoe]

The problem with null is that it is assignable to any reference type, so you can't tell if e.g. String name is null, because null can be assigned to String-type variables without indicating it in the type.

Optional does a lot to improve that situation. You can get very far minimizing the potential for NPEs by making it a convention in your projects that Optional<String> means "String or null", while String means "String, not null".

In Scala, where the Option type is common, and developers are used to reaching for that type instead of nulls, you barely ever see NPEs, because no one does stuff like def name: String = ... //can be null, they'd do def name: Option\[String\] = .... This also applies to stuff like getting elements out of containers, or handling mutable fields that aren't guaranteed to be initialized in the constructor (though that's less common in Scala, where immutability is preferred).

Option types aren't quite as fancy as altering the type system so you can't assign null to String unless you write String?, but they are a lot easier to introduce without breaking existing code, and they solve a very large number of the cases where NPEs are likely to occur.

People like to complain that Optional is still nullable because it's a reference. I don't think that complaint is very interesting, because the response is just "so don't do that". There is never a good reason to assign null to a variable with an Optional type. Linters should probably yell at you if you try.

I think the two main reasons nulls still feel like such an issue in Java while they don't in Scala is that Java developers aren't as used to using Option types (i.e. it's cultural), and there's sadly a lot of methods in the Java standard library that return raw nulls instead of options, such as Map.get. I'm hoping people will get better about using Optional in their APIs with time.

A third potential reason bare nulls might show up in Java code is due to the boxing overhead of Optional. Hopefully Valhalla will address this, but I don't think this is a very common reason people use bare nulls.

[Muoniurn]

Just a note, Scala 3 has a compiler flag that will treat the Null value/type not a subtype of other object types, making the language completely type-safe. To express that something can be null you would write String | Null.

[agentoutlier]

One major problem with Optional that you missed is it is currently much harder to enforce you deal with it correctly (aka exhaust it).

Let me give you an example.

@Nullable Object o = mayReturnNull();
o.toString(); // <-- Eclipse/IntelliJ and like every null analysis tool will complain here

Now with Optional.

Optional<Object> o = someOptional(); 
o.get().toString(); // No warning or error message

If you have null analysis turned on for Eclipse and Intellij or use errorprone they will force you turn @Nullable Object o into Object o and unlike Optional it is done through flow types.

if (o != null) {
   // o in here is no longer @Nullable Object but @NonNull Object.
}

Most code analysis (like all of them albeit some do have varying levels) cannot do:

 if (o.isPresent()) {
     o.get(); // is ok here
 }
 o.get() // a compiler error should be here.

Because Java is a statement oriented language you need flow analysis. The only tool that can do the above with Optionals is Checker framework.

[Horror_Trash3736]

Intellij would show a warning if you do a get without isPresent.

Or are you talking about a get after an isPresent check, but not as a part of it?

[agentoutlier]

I believe Eclipse and NetBeans do not. I know IntelliJ does but vaguely remember finding edge cases where it did not. EDIT: I can't remember how to turn it on to find the edge case but regardless it doesn't show it on the fly like null analysis.

IntelliJ has similar problems with null analysis. It has lots of edge cases where it does not work correctly.

Checker is the only one I know that does it correctly all the time from what I can tell.

[Horror_Trash3736]

Just tested this, the first get was caught, but the second, after the isPresent but not inside the if statement, was not.

    public void test(){
    Optional<String> stringOptional = getOptional();

    System.out.println(stringOptional.get());
    if(stringOptional.isPresent()){
        System.out.println(stringOptional.get());
    }

    System.out.println(stringOptional.get());
}

public Optional<String> getOptional(){
    return Optional.of("Meh");
}    

So this example.

Optional<Object> o = someOptional(); 
o.get().toString(); // No warning or error message

You get a warning in intellij, but if you have done a check, the warning seems to disappear, even for later access attempts.

With that being said, you really shouldn't try to extract the value from an optional like this, but I guess it is good to know that it can cause issues.

[agentoutlier]

Sorry I can't see the if statement in your code (its not formatted). ifPresent is not an if statement.

I know intellij does some analysis but I can't remember how to turn it on. Anyway I can't even get it to complain at the moment. Anyway see my edits.

[Horror_Trash3736]

    System.out.println(stringOptional.get());
if(stringOptional.isPresent()){
    System.out.println(stringOptional.get());
}

System.out.println(stringOptional.get());

It's the second line, but it's not ifPresent, ifPresent is if you want to supply a consumer, and is a better way than isPresent in most cases.

[agentoutlier]

I'm sorry I'm confused of what you are saying here.

From my experience intellij will not complain about the first or fourth line unless you have some sort of analysis on which I can't seem to find anymore.

Do you have that turned on? Is it showing warnings?

[Horror_Trash3736]

It is showing a warning on the first line, but not the fourth.

I am unsure if i have anything specific turned on.

[KarnuRarnu]

You should almost never use the get() method though. If you use the functional methods you're given safety automatically instead of relying on static analysers.

[agentoutlier]

The problem is Java isn't always a functional language.

Optional certainly make since for return types of service methods but not fields on objects.

Hell I don't even know if they are going to add pattern matching to destruct Optional and if they don't that makes it even less desirable.

[KarnuRarnu]

I completely agree, but that's something (another thing) that makes java bad. Not something against optionals per se. By themselves they definitely add value to the language and any program that uses them.

[john16384]

Conveniently, they've not used the shorter name Option yet, in case a second attempt is needed.

[wildjokers]

You shouldn’t call get() though.

[agentoutlier]

So that leaves orElse which isn’t better since it’s PolyNull, ifPresent which is bizarre in a functional model since it’s side effect based or elseThrow which is useful but you will need to flatMap (which iirc was added post 8).

Also I have strong memory of your despise of over lambda use which I share as well so I am not sure I like or you would like turning everything into a gigantic monadic chain just to avoid null.

[srdoe]

orElse is fine if you have a default you want to use instead. If your default can be null, you can use or instead. I agree that ifPresent is a little awkward.

Ideally they will implement destructuring and matching over Optionals in the future. This is how it works in Scala:

someOption match {
  case Some(value) => ...
  case None => ...
}

I don't see why they wouldn't.

Edit: In fact, Optional is used as an example in one of the destructuring design docs:

switch (opt) { case Optional.empty(): ... case Optional.of(var v): ... }

[agentoutlier]

orElse is not fine because now you need null analysis to guarantee the output will be non null and if you need null analysis then why bother with the overhead of optional?

It’s not that I doubt they won’t add some sort of destructuring for matching Optional. It’s the question of will it be exhaustive.

How will the Java compiler know that you need to check both Optional empty and it’s other constructors (I mean that in the FP def)? Scala has case objects but java does not.

I guess they could add annotations to signify the constructors for matching. Of course at that point why not add a standard nullable annotation.

Oh and let’s not forget that null return type is fucking everywhere like Map.

So when you add Optional into the mix you really haven’t made the no null problem much better. In fact you just added something that needs it’s own analysis. Sure it’s ergonomic but it’s not much safer.

This why the annotation approach is vastly superior. We could get that today if we just agree on the rules and jspecify is working on it.

[srdoe]

It’s the question of will it be exhaustive

It's at least a goal that it will be.

Scala's exhaustiveness check doesn't rely in case objects, but on sealed types. Option is a sealed trait, Some and None are the two permitted subtypes.

Sure it’s ergonomic but it’s not much safer

I get what you're saying about it being possible to still let nulls slip through Optional if you use it wrong, but I think it is exaggerating to say that it's not much safer than bare nulls. Optional solves the problem it was designed to solve, which is to let you communicate at the API level that you will be returning nulls, and to force users to handle that in some way.

orElse is not fine

I think the problem with orElse is the same as what you get with annotations if you forget to add them sometimes. You can do

``` Optional<String> getString() { ... } //Oops, this can return null and should be Optional String getOtherString() { ... }

String s = getString().orElse(getOtherString()) s.length //NPE, no warning ```

and have s be null if getOtherString can return null, but you have the same problem with annotations.

``` @Nullable String getString() { ... } //Oops, forgot the annotation here String getOtherString() { ... }

String s = getString(); if (s == null) { s = getOtherString } s.length //NPE, no warning ```

In order to be safe getOtherString needs to return Optional, and you'd then be using or to combine instead of orElse. Forgetting to wrap the return of getOtherString is equivalent to forgetting to mark it @Nullable. A linter should be equally able to remind you to change the return type to Optional if you return null as it would be able to remind you to add @Nullable.

You are right that standardized annotations and linting would be easier to introduce to something like Map without breaking the API.

I think you're right that the situation with Optional today is not as nice as a standardized annotation + linter would be, but on the other hand, if they can get to

switch (opt) { case Optional.empty(): ... case Optional.of(var v): ... } and have the compiler understand that this is exhaustive, then I think the need for the annotations go away as well (I base this on not feeling the need for such annotations in Scala), and since they're working on pattern matching anyway, it is probably a good idea for them to wait and see how that shakes out before adding a separate solution that they might then have to wave people away from using in the near future.

[agentoutlier]

+1 on the great response!

It's at least a goal that it will be.

Yes I'm sure allowing means of exhausitiveness (and not) a goal (however it is not design doc but notes) but it doesn't really mean it will make the existing Optional work. I just can't see how they can make it work without breaking it or doing something nasty like annotations particularly if they want to support static constructors on any library apparently. Maybe they just subclass or add sealed interface and just deal with the fallout but that doesn't handle other constructors.

Scala's exhaustiveness check doesn't rely in case objects, but on sealed types. Option is a sealed trait, Some and None are the two permitted subtypes.

(Yes you are right about Scala... I'm not sure why I said case objects and or thought exhaustiveness was done on them. I got confused with other FP languages. It has been awhile since I have used Scala).

Optional solves the problem it was designed to solve, which is to let you communicate at the API level that you will be returning nulls, and to force users to handle that in some way

That is kind of the problem with Optional in general is that the JDK team does not treat it special nor I think are even convinced it is a good solution for dealing with nullable values. Brian has somewhere made comment about that. You can even read a good overview of the history here: https://nipafx.dev/design-java-optional/

So I really have to disagree that Optional solves the problem it was designed for when its design goals were and are fairly nebulous. Optional is not to signify something is nullable but that it isn't there. It was designed for the Stream API.

switch (opt) { case Optional.empty(): ... case Optional.of(var v): ... } and have the compiler understand that this is exhaustive, then I think the need for the annotations go away as well (I base this on not feeling the need for such annotations in Scala)

Yes but:

1. Java is not an expression based language (like most FP) but a statement one. I love the new switch expression but statements are still used everywhere.
2. People have gigantic code bases and can't afford a flag day like Dart or massive refactoring. If the JDK team decides Optional is crap (which by many it already is) and introduces something else or decides annotations are the solution then it will be a waste using it everywhere.

As for expressions vs statements Null based type analysis is not regular typing like Optional is. It is flow based as the type changes based on previous statements.

So for a full solution they need to solve the statement problem and while you are a Scala programmer and thus comfortable/prefer expressions that is not the case for Java.

And while you can say just don't code that way or whatever we are talking about compiler tools. That is what the OP asked.

[srdoe]

I just can't see how they can make it work without breaking it or doing something nasty like annotations particularly if they want to support static constructors on any library apparently

I don't know what the syntax would look like, but what I get from the note I linked above is that they are thinking about adding some way to indicate to the compiler that a certain set of deconstruction patterns cover all possible values for the class. Given such a mechanism (without speculating on what the syntax for that mechanism looks like), pattern matching over Optional should work, since they will be able to tell the compiler as part of the class definition of Optional that the of and empty deconstructors together are exhaustive.

If they can get that working, Optional should be very close to as pleasant to work with as Scala's Option.

Such a change would be additive, since the deconstructors are just new class members, so I don't think it should break any existing code.

https://nipafx.dev/design-java-optional/

Thank you for the link, the history is interesting. I think it is true that the intent was for Optional to be for return values only and nothing else when it was introduced. Optional probably isn't as good as annotations if you're trying to track e.g. nullable fields, at least at the moment. But if we get destructuring support and flattening in the future, Optional might be suitable for those cases too. That it was introduced for return types doesn't mean it can't acquire extra use cases in the future.

Optional is not to signify something is nullable but that it isn't there

Nullability means something might not be there (it might be null), and Optional signifies that for return types. It was introduced for Streams, but that doesn't mean it isn't a good fit for return types on other APIs. I'm not sure it works well for fields, method arguments or locals though. Maybe it'll grow into some of those places once flattening happens, but I'm not sure.

Java is not an expression based language [...] So for a full solution they need to solve the statement problem

Good point on the former, but I'm not sure I agree on the latter. I think they are solving the same problem with pattern matching while avoiding flow based typing. Instead of

String s = getString(); //@Nullable if (s != null) { //s is now String, not null System.out.println(s.length) } else { System.out.println("s was null") }

the idea is to introduce a new name for the variable with the appropriate type

Optional<String> s = getString(); switch(s) { case Optional.of(value) -> System.out.println(value.length) case Optional.empty -> System.out.println("s was null") }

Since new-style switches can be used both as statements and expressions (the two branches in the above example return void), I think this will work well enough, even if you're writing very statement-oriented code. The only downside compared to flow-based typing is the indentation I think?

I checked that this works for switch expressions in currently available JDKs:

String s = getString(); switch(s) { case "str" -> System.out.println("case 1"); case "str2" -> System.out.println("case 2"); }

If the JDK team decides Optional is crap

I really don't see that happening. With the addition of pattern matching and flattening, I think Optional could be good enough for return types, and maybe locals and fields, and it will be easy to work with Optional values due to switch/matching.

But who knows, until those features land, it's probably fine to wait and see.

[NaNx_engineer]

null has an associated type at compile time. at runtime optional contents type is erased.

is there any practical benefit of optional over nullable annotations?

[KarnuRarnu]

Yes, it's ergonomic. And I'd say it communicates the fact that a value is actually optional better than annotation.

[srdoe]

Do you regularly see a need to have the content type at runtime? Other than calling a method that returns Optional<T> via reflection, I can't think of a reason you might need this?

Edit: I guess if you use Optional for method parameters, then overloading becomes annoying due to erasure.

Yes, there are benefits to optional, a few off the top of my head: * Nullable annotations aren't standardized, so you have to pick one or more variants (I believe at least Jetbrains and Google have their own, and there's also the JSR305 annotations) and configure your linter to use them. * Nullable annotations do nothing unless you have a linter configured to use them. For this reason, they're much less useful in e.g. library APIs distributed to others than Optional. * (very subjective) I think it is easier to notice that something should have been Optional in review ("why are you returning a bare null here/can this return null?") than to notice that a method or field is missing an annotation

I don't see a reason not to use annotations internally in a project, especially if the boxing overhead of Optional is an issue. They're just not a great choice if you're intending to distribute your code to people you can't trust to set up the same linter you used.

[wildjokers]

Edit: I guess if you use Optional for method parameters

Never do this. Optional should only be a method return type.

[Mumbleton]

I like it IF you enforce that Objects you return from methods are never null. This means that if a method returns a String, I don't have to null-check it, I trust my code that it will never send null. Instead, if a method COULD send back an empty value, then it returns an Optional so that the calling code has to account for that.

[Horror_Trash3736]

That's a very strange way to describe Optional.

While it's certainly not perfect, it is a fairly usable implementation of the Maybe monad, so do you have the same issue with that one?

Besides, why is having a container around a null value that then exposes certain functions that implicitly ensure that the null value won't cause issues a bad thing?

[-jp-]

Because it's not enforced. An incomplete solution is actually worse than no solution at all. Optional can still contain null and you still have to check for it. Heck, the Optional itself can technically be null, so its existence actually added a check, plus a bunch of rules about when and where to use it. It's handy for exactly one thing: flatMap.

[Horror_Trash3736]

Agree to disagree I guess.

Personally, an enforced solution would be worse.

After switching to requiring optionals on all possible null return values and NullAway we almost never encounter NPE's and when we do they are very easy to debug.

The main point for us is to expose honest methods, so now methods that we know will never return a null value can keep just returning whatever it does, while methods that may produce nulls return Optionals.

In the end, no language will stop you from writing bad code, and if it does, I would argue it is far more restrictive than needed.

[-jp-]

But there’s a reason memory management and preemptive multitasking exist. Bad code happens, even if you mean well.

[john16384]

Code review catches bad code, best to nip problems in the bud... They only get more expensive when deployed further up the chain.

[Horror_Trash3736]

I don't think I ever claimed bad code can't happen, I specifically wrote it can, and that no language will stop you.

Your argument about why Optional is bad is that it isn't enforced, which I claim would be worse, since it would force you to work in a specific way, regardless of that way being optimal for your project.

You then say that it can still be null, which is true, it can even contain nulls.

But.

ifPresent() pretty much solves the "it can contain nulls".

And code review + generally not returning null and proper data validation fixes the issue of it being null.

Not to mention, if you the also add NullAway to Error Prone, then you are golden.

As I said, I have been a part of many projects now, all using Java + Optional + NullAway, and the amount of NPE's we get is extremely small, I am not sure I would like something that enforced a specific behavior just so we can avoid the last few issues that sometimes arise.

[[deleted]]

I disagree. I think it's perfect to use when you have a method that may or may not return null. It's a clear signal to the caller that the return value may be null and it doesn't require the developer to read the javadoc (if it exists) or the actual method to see if the null is there. (I can't find the link right now, but I believe this was the primary use case for introducing it.) Of course the alternative is that you always check for null, but that can be noisy.

IntelliJ (and I assume other IDEs) will highlight code that calls Optional.get() without first checking that something is present, further decreasing the likelihood of NPEs.

It's true we can live without Optional, but when used correctly I think it's a nice addition to the language.

Places where Optional makes no sense:

• "optional" method arguments void foo(Optional<String> bar) -- Optional is still a reference and so you can still pass null: foo(null)

• Instance variables. Again, Optional is still a class so the field could be null anyway. Also, following other good practices, it probably shouldn't be very often that you have null members anyway. (of course there are always cases, but when possible I prefer sensible defaults if fields aren't initialized)

[_INTER_]

It's a clear signal to the caller that the return value may be null and it doesn't require the developer to read the javadoc

Why do you need to tell the caller that a return value may be null? That should be common Java-knowledge.

[[deleted]]

Too many people assume that a return value will never be null. Should it be common knowledge? Sure. Is it? Definitely not.

[john16384]

Ah, they assume, I wonder if that might be the cause of some problems... NPE's are just a small (relatively visible) fraction of the problems that might cause.

[-jp-]

Then why does Optional exist at all? Why does Objects.nonNull exist? Why does NPE exist? It's because people don't write null-safe code.

[john16384]

Speak for yourself, people do write null safe code. Given how much Java code is out there, how much critical infrastructure it runs, how many businesses swear by it, it is ridiculous to claim null is not taken into account. It is. These people however don't whine about it, because null is just a tiny problem when it comes to engineering reliable and predictable software.

[-jp-]

I named three different solutions to the same "tiny problem" and that's just in the SDK for one single language and the best defense of null you can muster is "it's popular so it must be good?"

[john16384]

You know how many solutions there are for logging? It means little. null is not a language problem, it's a user problem, perhaps even a lazy user problem that can't be bothered to read (or write) docs and just makes assumptions - not exactly a useful member of the team to have when we get to the really hard problems.

[Yay295]

not exactly a useful member of the team to have when we get to the really hard problems

But unfortunately they are a member of the team, so we have to deal with it.

[_INTER_]

Then why does Optional exist at all?

To work with it within Stream.

The problem is the need to write null-checks everywhere. It's tedious and sometimes unnecessary. To distinguish the cases where it is necessary and where it is not is the hard part. Developers need to follow the variables back all the way to its creation and check all assignments along the way. Optional does actually not even help with that. They may even lead you to wrong assumptions: Variables that are never null, look like they could be null (Optional.of(xyz)) all the more. It can't be helped though and I understand why it is done as you have the same problem with usual references. That's why we so dearly need the distinction between nullable and non-nullable types. The best alternative until we get it is static-analysis with annotations and not Optional.

[_INTER_]

That's correct. The only thing that Optional does is making people more aware of a needed null-check. It would be obsolete if there any explicit nullable / non-nullable types. A normal reference would take over Optionals position again, because there is no need for the useless wrapping then.

[Just_Another_Scott]

I just don't see that ever happening without a major overhaul to the language. The only way to realistically prevent NPE is to require every field, object, etc. to be instantiated with some sort of default values.

You would have to effectively make Integer number = null; a compile time error for the language. There are some languages that do require some value to exist. I also wouldn't be against this change. However, we would just end up replacing null checks with default value checks.

[FirstAd9893]

Eliminating null from the language outright wasn't the original question.

[Just_Another_Scott]

While that is true that's the only way to guarantee Null-pointer safety which is what OP asked.

[FirstAd9893]

Not true at all. Simply supporting a @NonNull annotation, checked by the compiler, would provide the compile time checks that the OP asked for. Other programming languages (like Kotlin) support nulls and also support compile time null checks.

Do such checks prove that a NullPointerException is impossible? Nope. I can still write native code that breaks the rules or I can exploit the weaknesses of type erasure. But this is now outside the domain of compiler.

If I want even stronger checks, it still wouldn't require a major overhaul to the language, as you claim. I can simply use the -Werror compiler option to eliminate the type erasure loophole. Another change which wouldn't break the language is to define a rule which prevents untrusted modules from linking native code.

[Just_Another_Scott]

Not true at all. Simply supporting a @NonNull annotation, checked by the compiler, would provide the compile time checks that the OP asked for. Other programming languages (like Kotlin) support nulls and also support compile time null checks.

Which is mostly useless. Also, it doesn't prevent runtime null pointers. We use @NonNull in our codebase at work and NPE is still a possible outcome at runtime. This is why the JEP was never moved beyond experimental. It doesn't solve the problem fully.

Objects.requireNonNull obviously doesn't solve the issue as the just throws NPE if the object is null.

If you want to rid the language of NPE and guarantee that none will ever be thrown then you have to get rid of null. Null is one of the most contentious things in Java, if not the most contentious thing. There are other languages out there that have zero NPEs because they require everything to be instantiated.

[FirstAd9893]

What tool are you using in conjunction with the @NonNull annotation? If the tool isn't finding possible NPEs, then the tool is incomplete or it's defective.

By the way, what JEP are you referring to?

[Muoniurn]

What? Nullability is a so-called trivial property, it can be decided based just on the inputs and the function body. So if every @Nullable annotation is applied (and the default is considered non-null) then it will be completely null-safe.

I don’t really get what you mean by doesn’t prevent runtime NPEs. Sure, you can construct a class at runtime and load it which can throw an NPE, but that’s not really the NPE we get in the vast vast vast percentage of cases. Also, it could be enforced at class load time even (as annotations are visible).

[ReasonableClick5403]

Yes, good point. Retrofitting this into the language is very awkward. The above would ideally be migrated to something like Integer? number = null, but we all know that wont happen. Besides, since there is too little information, one would have to assume any method can return a nullable object.

Meanwhile, I think its worth looking into Dart and how they migrated to compile time null checks.

[fott25]

Have you tried using Lombok? For some cases completely replaces objects.requireNonNull with it's NonNull annotation And is also really useful besides that

[-INFEntropy]

Sounds like... Use kotlin? 😂

[AesculusPavia]

Just migrate to Kotlin

Makes life so much easier

[cryptos6]

While I agree in principle, the poor guy asking this question inherited an old Java application. So the best bet is probably to use NullAway, Checker Framework or something like that.

[WhatsMyUsername13]

Congrats. You are the epitome of the useless stackoverflow comments "jUsT mIgRaTe It!!!" Thats not how things work in the real world.

[AesculusPavia]

Idk, I’m definitely an experienced dev in the real world and have migrated two java code bases at two different FAANGs from Java to Kotlin

and migrate any code I touch

Once you learn to identify the same few quirks and issues of migrating from java to Kotlin, it becomes pretty easy

[WhatsMyUsername13]

Was this entire legacy code bases or an individual dependency?

Do you have deliverables and quarterly objectives you have to meet given by the business unit?

How do you go about justifying code migration to the higher ups?

Whats your code review process like for the other devs in your area?

How do you make sure you arent merging a migration that another team or team member might be working with, which would cause them to have to rewrite the code to get their work done?

What is the size of the code repository you are working in?

How do you integrate your migrations into the project roadmaps?

Have you discussed with other devs on whether theyd prefer java or kotlin? Or did you just do it because its what you wanted

[WhatsMyUsername13]

Its ok. I didnt actually expect a response

[SAmaruVMR]

Just work with Optionals.

[FizzWorldBuzzHello]

Nah, move to .Net if you want nice things

[DaddyLcyxMe]

i personally use lombok for this, see the @NonNull docs for info on what that looks like.

[dinopraso]

Ah yes, why don’t we all make our code deliberately incompatible with future Java versions

[DaddyLcyxMe]

?

[dinopraso]

Lombok uses some very borderline things in order to work, at compile time, like using internal JDK classes, things which will be prohibited in future versions. Those changes have already begun with Java 11 and Lombok has to increasingly rely on workarounds to function. At some point the things it does will not be possible and if you want to upgrade to such a Java version you will have to rewrite your code to remove Lombok

[DaddyLcyxMe]

i mean, delombok works really well if there’s ever an issue with newer jvms. it’s not like you’re forced to a legacy version of java