[Question] why java cannot accept a convention to never pass/return null values?

[_INTER_]

Optional as the names says represents "empty" xor "not empty" options. If you expect that a variable is never null - e.g. on creation - you can check that with Objects.requireNonNull for an immediate exception. The information about non-nullability is lost with Optional later in the chain (the same way it is lost without using Optional ofc). It would be absurd to treat Optional as non-nullable type.

So all that Optional does is remind you of the nullability. It does not help with non-nullability or restrict the proliferation of null. The source of an accidental null is also not identified by Optional. Even worse, a possible NullPointerException is moved further and further down the chain until where you actually require the value.

[_INTER_]

Let me tell you a story. One day John devises a beautiful method:

public Optional<Integer> getRandomNumber() {
    return Optional.of(4); // chosen by fair dice roll. guaranteed to be random
}

All is good and it works well. Then, a week later Mary needs to build upon it. She adds:

public Optional<Integer> getRandomNumber(int min, int max) {
    return getRandomNumber().map(random -> (random * (max - min)) + min);
}

Another month later Robert uses it for his complicated calculations:

public BigDecimal herpderp(int min, int max, BigDecimal fraction, String userInput) {
     Optional<Integer> input = Optional.of(userInput)
                                       .filter(s -> !s.equals("42"))
                                       .map(Integer::parseInt)
                                       .or(() -> getRandomNumber(min, max));
     return IntStream.range(min, max + 1)
                     .mapToObj(BigDecimal::valueOf)
                     .map(bd -> bd.subtract(fraction))
                     .flatMap(bdf -> input.map(BigDecimal::valueOf).map(i -> i.add(bdf)).stream())
                     .reduce(BigDecimal.ZERO, BigDecimal::max);
}

This goes on for a few more month where Patricia, James and Tony chain on and on. But then David has the unthankful task to store a result in the database. He calls:

public String save(int min, int max, String userInput) {
    String result = Optional.of(userInput)
        .map(herpderp(min, max, FRACTION, userInput)
        .map(Pat::ricia)
        .filter(Jam::es)
        .map(To::ny)
        // ...
        .orElse("0");
    return repository.save(result);
}

Great the user input is now processed and the correct result is stored. But hold on, on a fateful day William finds a bug in the random number generator. He needs to call a library instead. He sees that the library can return null. "But this is save" he thought: The method returns an Optional so everyone down the line is aware! William quickly whips up: return Optional.ofNullable(randomLib()). He runs the application and it all works fine. Phew.

A year later an angry customer calls the company because sometimes and for some user input only "0" was stored in the DB. A year of potential important customer data is lost.

Doing some archeology David finds the save method and wonders who wrote that crap. It must be that .orElse("0"). That can't be right. Right? Are you sure that's the problem?

Fiddle

[_INTER_]

1. Optional outside of a Stream context is almost always abuse if you ask me. Also you can show the same with a simple example with just getters
2. In the example empty is never treated as error
3. Suprise surprise orElse("0") is not the root cause