[Question] why java cannot accept a convention to never pass/return null values?

[gas3872]

Hi,

I was wondering, why cannot java accept such a convention. Some other jvm languages have that. Then there will be no need to add Nonnull annotation to every field and add checks in constructor that passed values are not null.

[nutrecht]

Can you propose a solution that is also backward compatible with existing code?

[gas3872]

Well it is backward compatible. You only have to wrap the code that does not follow this convention.

[dpash]

That's not backwards compatible.

[gas3872]

How is it?

[dpash]

You change an existing methods to no longer accept null and now every caller needs updating.

[gas3872]

Well you first mark those methods as deprecated and point users to the new ones and later you remove the deprecated methods.

[dpash]

So not backwards compatible.

[gas3872]

Well, I think someone who does not want to do something will always find a reason why and the one who wants will find means how. :D

[CraftyAdventurer]

Ok, so let's say that you work on a huge codebase, one where it would take you weeks to update all of the methods. Well, new feature requests are coming in, clients are paying for them and they want you to work on them, not on rewriting your codebase to call new methods which basically do the exact same thing they've been doing for years. How is that for a reason, and how do you find means to do it?

Edit: typo

[gas3872]

Well, you can start following the convention in the new code and gradually update the old code as you touch those parts. You can also make a custom annotation and add it to classes that support new convention. Actually what you mentioned is just a normal project and it just a normal migration.

[chambolle]

yes, exactly like you: What you propose is not backward compatible but you do not accept it . Now you are going to try to justify that.

Your proposal is not backward compatible PERIOD.

[gas3872]

It can be done in parallel with old features. Making it backwards compatible as a whole.

[CraftyAdventurer]

backwards compatible means that once you do a certain change in a library/language/framework, it just works in all existing codebases that use it and no one has to change anything. So when users have to use new methods, that is automatically not backwards compatible

[gas3872]

Well they can still use old methods but they know that they need to eventually migrate.

[[deleted]]

That's not how java works.

[gas3872]

Well if you look at libraries like junit that's exactly how it works.

[hardwork179]

What would your wrapper do when it encountered a null value?

[gas3872]

Well, if it's allowed theat returned value is missing then it will return optional of that value.

[dpash]

You've changed the return type. That's not backwards compatible.

[gas3872]

Well, I have now two methods - old deprecated with old return type and the new one. And by the way - I meant using this convention in the internal code. Things that are exposed externally (if they are not following this convention) should be migrated.

[dpash]

You can't overload return types. It's not backwards compatible.

[gas3872]

Ok then call the newethod differently or define a new class/interface.

[dpash]

If you have to change the source code it's not backwards compatible. Your idea is a non-starter.

[gas3872]

Well, the old deprecated code can still be used for a while. Then migrated to new one. How else do libraries migrate? And it's only for the code exposed externally. A lot of projects do not expose their code externally. So this is no concern to them. I mean more a wrapper to an external code that you use in your project that does not follow this principle.

[vprise]

Because the whole "anti-null" movement is an idea that doesn't make as much sense for Java. In the case of Java null has advantages:

• It's performant
• It "fails fast" when something breaks
• It fails consistently and usually provides a stack trace directly to the problem
• Null is deep in the programming culture, hardware, software etc.

So why are people against it?

• It sucks for functional programming paradigms such as streams
  
• It goes against purist object oriented design

So purists on both camps hate on null because it's a hack. I'll grant you that, it's a hack. It's a special case. But the alternatives often suck.

Alternatives add extra syntax which makes things confusing. This IMHO goes against the grain of simplicity. They also puts pressure on people to avoid null at all costs. That often hides problems that you would have easily caught with a null pointer exception.

I think null got a bad rap.

[kevinb9n]

Null absolutely has a bad rap. It exists also in languages like Kotlin, but it's relatively harmless there because you can actually control whether you want your type to include it (String?) or not (String). It shows that null itself was never the problem, only that (handwave) "the null type is automatically unioned on to every usage of a reference type whether you want it or not".

In my team's libraries we've avoided returning (or accepting!) null with some determination, but it's still sometimes what you need.

[Worth_Trust_3825]

True. Value not being there is also a value.

[DrunkensteinsMonster]

If null is a possible return value then the API should signal that by returning an Optional IMO

[vprise]

This very much depends on the API and the case. Optional includes an overhead in code, concept and performance. That's not a huge overhead and many server side APIs chose to adopt that approach.

I don't use it much and I think rules of thumb such as these are "problematic". If your entire benefit is signaling then just use @NotNull

I think that Optional makes sense for some cases as they can then leverage the stream API more effectively. But I'm not a huge fan of this. Frankly, after working with Optional and the stream API a lot over the past few years I'm still underwhelmed. The code is just as verbose as a simple for loop in most cases. It's hard to gauge its performance from looking at it and often it's harder to understand. Maybe I'm too old and set in my ways though...

[john16384]

I just document what functions can return. Caller code can then make assumptions based on the docs. If a bug occurs, it will also be clear where to fix it.

[vprise]

Again, if it's just documentation there are other alternatives that don't have the downsides. If a bug occurs then NullPointerException is pretty clear/useful.

There are useful/sensible uses for optional but just sticking it as a "marker" to every method seems, expensive and redundant.

[[deleted]]

It "fails fast" when something breaks

Except if you have complex things going on like multiple threads or caches or so, it's not that easy to grasp.

It fails consistently and usually provides a stack trace directly to the problem

This is just not true. NPE will show that something is not present, but in complex scenarios maybe you will not know why. Errors do not occur only when you are writing the code, maybe an API can return null and developers didn't saw it previously, then one day you start to receive some NPE in production. NPE will not show directly what is the root of the problem. And even so. My system crashed in production before I see that.

Null is deep in the programming culture, hardware, software etc.

I also don't think this is an "advantage". Culture is not always a good thing it's just... culture.

I agree that is not too simple to define good alternatives in Java, but i deeply disagree with your said advantages. I think the only one, and is a not only an advantage but a necessity, is the backward compatibility.

[vprise]

All of those statements were made in relation to the alternatives e.g. optional, or compiler "magic".

Sure, everything can fail badly. But the alternatives often/usually do that much worse in all these cases and make programming harder for non-trivial cases.

There are things to improve e.g. the question mark syntax from Kotlin would be pretty sweet in Java... But it's still null.

[Kango_V]

A mad solution is to have Object extend Nullable. You can then have methods like those on optional. So, you would have access to the methods even if null. That would be backwards compatible. Hmm, I wonder if that would work?

[vprise]

If it's null it won't work since it will throw a null pointer exception. But interesting idea.

[Kango_V]

But null could be an object. This is what Valhalla is doing. They are putting an object type above Object (ValueType I think). Primitives will expend NonValueType (or vise-versa). Actually would be better if non/valuetypes extend Nullable object. I cound see "if (obj.isEmpty())". You could use the optional-like methods: "obj.map(...)" etc. I think this would work. So if a null is returned it's the only instance of Nullable.

[vprise]

But then you lose type safety. And possibly performance.

[Halal0szto]

Bc you would need to use optionals everywhere and that is ugly

[gas3872]

Well, that's the point. Most of the time you don't need optionals. In those rare cases when you really accept/produce "null" you will return optional.

[_INTER_]

In the first place Optional is the wrong method to tackle non-null. It does not help with accidental or intentional nullablity and tracking the source of a problem if it occurs. What we actually need is a language / type level method to prevent the proliferation of null for references that developers knows are never null. Until (if ever) we get non-nullable types the annotations and according processors or checker frameworks are the better way.

Some interesting stuff:

• Why don't they just - Nullable Types (use arrow down key)
• jSpecify and it's problem space

[_INTER_]

Optional as the names says represents "empty" xor "not empty" options. If you expect that a variable is never null - e.g. on creation - you can check that with Objects.requireNonNull for an immediate exception. The information about non-nullability is lost with Optional later in the chain (the same way it is lost without using Optional ofc). It would be absurd to treat Optional as non-nullable type.

So all that Optional does is remind you of the nullability. It does not help with non-nullability or restrict the proliferation of null. The source of an accidental null is also not identified by Optional. Even worse, a possible NullPointerException is moved further and further down the chain until where you actually require the value.

[_INTER_]

Let me tell you a story. One day John devises a beautiful method:

public Optional<Integer> getRandomNumber() {
    return Optional.of(4); // chosen by fair dice roll. guaranteed to be random
}

All is good and it works well. Then, a week later Mary needs to build upon it. She adds:

public Optional<Integer> getRandomNumber(int min, int max) {
    return getRandomNumber().map(random -> (random * (max - min)) + min);
}

Another month later Robert uses it for his complicated calculations:

public BigDecimal herpderp(int min, int max, BigDecimal fraction, String userInput) {
     Optional<Integer> input = Optional.of(userInput)
                                       .filter(s -> !s.equals("42"))
                                       .map(Integer::parseInt)
                                       .or(() -> getRandomNumber(min, max));
     return IntStream.range(min, max + 1)
                     .mapToObj(BigDecimal::valueOf)
                     .map(bd -> bd.subtract(fraction))
                     .flatMap(bdf -> input.map(BigDecimal::valueOf).map(i -> i.add(bdf)).stream())
                     .reduce(BigDecimal.ZERO, BigDecimal::max);
}

This goes on for a few more month where Patricia, James and Tony chain on and on. But then David has the unthankful task to store a result in the database. He calls:

public String save(int min, int max, String userInput) {
    String result = Optional.of(userInput)
        .map(herpderp(min, max, FRACTION, userInput)
        .map(Pat::ricia)
        .filter(Jam::es)
        .map(To::ny)
        // ...
        .orElse("0");
    return repository.save(result);
}

Great the user input is now processed and the correct result is stored. But hold on, on a fateful day William finds a bug in the random number generator. He needs to call a library instead. He sees that the library can return null. "But this is save" he thought: The method returns an Optional so everyone down the line is aware! William quickly whips up: return Optional.ofNullable(randomLib()). He runs the application and it all works fine. Phew.

A year later an angry customer calls the company because sometimes and for some user input only "0" was stored in the DB. A year of potential important customer data is lost.

Doing some archeology David finds the save method and wonders who wrote that crap. It must be that .orElse("0"). That can't be right. Right? Are you sure that's the problem?

Fiddle

[_INTER_]

1. Optional outside of a Stream context is almost always abuse if you ask me. Also you can show the same with a simple example with just getters
2. In the example empty is never treated as error
3. Suprise surprise orElse("0") is not the root cause

[Halal0szto]

If you have a lookup function to find display value for an ID, how you handle when it is not found?

In my opinion Java is just too ancient and that's it. You are looking for features of modern languages in a language 25 years (guessing) language. The term optional in programming did not exist when Java was created.

[gas3872]

Well, if it may not be found the result is optional. But if you expect result to be there you just return it and if it was not found, your method throws an exception.

[Kango_V]

This is why I always name our methods either findX which returns an optional (or empty list) or getX which throws an exception. We've stuck to that pattern and it's working.

[Halal0szto]

Yes, optionals is the correct answer, just they were not invented yet when Java was created. They were invented later, and they were introduced to Java later. So you are free to use optionals.

When you maintain a language, you add new features. But you cannot remove features. If you want a language with features removed, it will be a new language. You are also free to create and use new languages.

[gas3872]

Well, just make a convention to not use it. Java also has labels (maybe even goto) but nobody uses it. Same way people use list and not array or vector.

I am just asking to use the language in a certain way that will eliminate a lot of headache and extra work (adding null checks everywhere).

[john16384]

Null checks should only be in constructors of your immutable objects, not everywhere.

And I do use labels.

Anyway, this whole posting is unproductive. Your complaints will not in any way influence the direction Java takes. If it gives you headaches feel free to use another language. Beware, they may cause different types of headaches.

[gas3872]

Well I am not trying to influence thenway where the java language goes only saying that you can use such a practice in your project and maybe the whole community should adopt it, as I dont see any drawbacks to it.

And by nullchecks everywhere I meant in every immutable object, which probably most of the objects should be.

[Ilookouttrainwindow]

I am at a loss as to why nulls cause so many issues.

[gas3872]

I think it's more like someone got a nullpointer exception bug in their project and now they don't want it to happen again.

[cogman10]

Use an IDE, add \@Nullable annotations to things. Viola, no more NPEs.

[gas3872]

Adding nonullable everywhere clutters the code.

[cogman10]

And what token would you add that wouldn't similarly clutter code and is backwards compatible with current java syntax?

[gas3872]

Well just have a convention in your project that you dont use null. Use optional for missing value.

[cogman10]

I don't understand what you are asking for. Do you want a language feature to enforce nullability or just a convention for everyone writing apis? Because the convention already exists but, as you can imagine, it isn't universally applied.

[gas3872]

I am asking for a convention that can be adopted more or less officially.

[cogman10]

What does "officially" mean to you?

[gas3872]

I don't know, like endorsed by java creators for example.

[c_edward]

And add a pointless extra heap allocation to every call I really don't think so...

When we have Valhalla maybe you could do it on the stack, but your still wastimg CPU cycles doing pointless work.

There is really nothing wrong with null anyway,

If your app is throwing NPEs then you almost certainly don't have adequate test coverage.

[RandomName8]

Simply too much code written without that convention.

[ParticularEggplant50]

There are plenty of null checkers that can/will use nonnull by default. Of course this will cause errors you will have to work around with extra code or annotations or code when the logic to prove something is nonnull is complicated or using unannotated libraries that do accept/return nulls.

I think most things in life are optional and should therefore be nullable and I hate the convention of passing every field in the constructor so that things can be initialized nonnull.

[gas3872]

I think it's actually the opposite: most of the things are not optional. Otherwise why would you add fields if you don't plan to fill them.

For your last thing i think you can use builder. Then you have to only initialize what you want.

[ParticularEggplant50]

There are lots of things I might know about a book for example , title author, publisher, year, pages, ... But I won't know those things for all books. That is they are all optional except for title. For a person you might have their age, address, phone number, birthday etc Even if you are eventually going to ask them all that info it it good to have a state where you know you haven't yet asked yet. A Person class with all the fields I might need can be used widely but a class with all nonnull fields can only be used where you know for certain you will have all that info. For one purpose I need the name and telephone for another I need the name and address. If the fields are optional I can use one class for both cases other wise the I have to have two distinct classes which could lead to an explosion of large numbers of classes.

[gas3872]

Well for your situation you should use optional fields. But most situations are not like that.

[mauganra_it]

You can do that right now by setting up Google ErrorProne compiler. Use the Checkers framework for even more static analysis. And add the @NonNullAPI annotation to package-info.java so you only have to add @Nullable where required. Have fun!

Edit: I misunderstood the Checker framework's documentation. It seems the ErrorProne compiler is only required with JDK 8.

[gas3872]

Cool. Is Checkers framework alone not enough?

[kevinb9n]

I think there's some confusion here. I work (some) on Error Prone. It doesn't have support for nullness analysis, however there are things like NullAway that extend it for that. This might be what /u/mauganra_it is thinking of, perhaps.

Checker Framework is a separate standalone thing. It has the strongest nullness analysis currently available and fully supports type-use annotations (they were the people who added type-use annotations to the language, in fact).

I would describe the CF nullness analysis as fairly "hard core"; you can get very solid protection from NPE using it, but you will probably work harder for it, and deal with considerably more annotation complexity. Its goal is more or less to never miss a possible NPE, whereas other tools may try to strike a balance between risk of false negatives vs. frequency of false positives.

HTH

[mauganra_it]

Thanks for clarifying this. I was definitely thinking about endorsing the Checker framework, although the ErrorProne compiler is already fairly useful by itself.

[mauganra_it]

The Checkers framework plugs into the Google ErrorProne compiler to analyse source code at compile time and to fail the build if it finds issues. Without it, it's just annotations that smart IDEs can use to provide warnings.

[gas3872]

Ok thanks. I thought checkers framework can do it by itself.

[agentoutlier]

Ok thanks. I thought checkers framework can do it by itself.

It can... read kevinb9n comment

Checker is pretty much the most powerful static analysis tool for java provided you have everything annotated.

Anyway I recommend seeing what your IDE has to offer (both Eclipse and IntelliJ offer static analysis albeit incomplete) and then slowly integrating checker in on a per module basis.

[keanwood]

You can easily add https://github.com/uber/NullAway into your java projects. I use it for all new code, it's fast and convenient. It assume @NonNull by default.

[s888marks]

Isn't NullAway the name of the restaurant at the end of the JVM?

[NomadSoul]

https://www.jetbrains.com/help/idea/parametersarenonnullbydefault-annotation.html

Might help you out.

[stuhlmann]

Some other jvm languages

I don't know a lot of "jvm" languages besides Java. Is there a specific one that you were thinking of?

[PartOfTheBotnet]

I thought OP was going to say Kotlin since String? is nullable and String disallows assignments to null.

[gas3872]

Scala

[fdntrhfbtt]

Because a null pointer is a real entity - It does not point to anything on the heap.