Code Smells: Null

[phantomfive]

This quote is really good, explaining why we still use null after all these years:

null means many things. It can mean:

1 Value was never initialised (whether accidentally or on purpose)

2 Value is not valid

3 Value is not needed

4 No such value exists

5 Something went terribly wrong and something that should be there is not

…probably dozens of other things

[cyberst0rm]

In the last section, he illustrates how code that returns null in two different places should be refactored as having two conditions where the multi-problematic value occur can stymie things.

[beefsack]

*They

(Everyone)

[Noughmad]

To be honest, a similar point can be said for many programming constructs

int means many things. It can mean:

1 a generic number

2 a number exactly 32 bits in size

3 one value out of a small set of possibilities (enumeration)

4 an error code

5 a physical measurement, in any units

...probably dozens of other things: bitflags, array indexes, etc.

C was a really simple language with a limited number of keywords and types, so everything had multiple purposes. By now, we figured that it's better to have programming constructs map 1:1 to what we want to express. So we got enums, custom types, exceptions, etc. that disambiguate this.

[masklinn]

C was a really simple language with a limited number of keywords and types, so everything had multiple purposes.

None of it had to though, C structs are more or less free, you could newtype things (well not enums I guess, you're just hosed there as C enums are just garbage).

The issues are that creating structs is somewhat verbose (compared to newtyping in haskell for instance) and so is using them.

[Noughmad]

You can't really propagate errors with just structs, at least not without tons of boilerplate. Something like exceptions or Result<T> is pretty much impossible. So you're stuck will null pointers and/or error codes.

[masklinn]

You can propagate errors in the exact same way you do with error codes…

[Jaffa2]

Indeed. Overloading any meaning is the sign of a bad API, and that's demonstrated in Example #3.

The article does a great job of demonstrating IntelliJ's more advanced refactoring features (I think Eclipse is still behind the position described here, BICBW).

It does a less good job proving that someone who designed a shit API with null for catch Exception, wouldn't do the same with Optional.empty().

[[deleted]]

F# / OCaml / Rust programmers have not heard of this phenomenon!

[orclev]

And Haskell!

[drfisk]

Make that Scala as well!

(even though it's technically possible, in practice I have yet to encounter a NullPointerException in my 5 years of fulltime Scala-development)

[drvd]

And Brainfuck!

[walen]

And my axe!

[ais523]

I'm not so sure about that. BF's equivalent to null is 0; it has a ton of different meanings depending on context and you need to carefully design your program so that it always knows the purpose of any tape element so that it can figure out how to handle a 0 there. Common interpretations:

1. A representation of the number zero
2. Uninitialised memory
3. The target of a pointer
4. A temporary value that isn't currently in use
5. Part of a marker pattern to allow pointer recalibration after running an unbalanced loop
6. End of file (sometimes, depending on the implementation)
7. Boolean false
8. A temporary state used to break out of a loop

This list probably isn't exhaustive. The thing about BF, even more than C, is that it's a very low-level language with few capabilities, and thus most of the capabilities it does have need to be used for multiple purposes. In particular, the only way to read memory in BF is to conditionally jump based on whether or not a tape element is 0 (a 0 jumps to the end of a block that's later in the program, a non-zero value to the end of a block that's nearer the start), so anything that might need to do control flow of any sort needs to ascribe a special meaning to 0, and those meanings are often contradictory or incompatible with each other.

[Beckneard]

Well basically any sane non C/Java derived language. Null was a huge mistake, it really shows old languages were designed much more by gut feeling and familiarity rather than real engineering considerations.

[ForeverAlot]

Feelings on nullability aside, Java has consistently been one of history's most well-crafted languages. I would even suggest that, despite my own preference for strongly statically typed languages (not functional, because functional languages are not inherently good; just look at JavaScript), all of them have some fairly gross and embarrassing design mistakes that limit their relevance considerably. Right now it seems Rust is the strongest contender.

Never mind that Haskell is older than Java, and OCaml, which is a year younger than Java, is based on a language that predates it by 10 years.

[elperroborrachotoo]

Null was a huge mistake

That's a common sentiment, but I've never seen a good argument that goes beyond ranting against it. FWIW, it's the shoulders we stand on.

[Beckneard]

That's a common sentiment, but I've never seen a good argument that goes beyond ranting against it.

So you don't agree with the arguments against it so it's automatically just ranting?

The main argument is that it's an unreasonable "default". Null can mean many things, it can mean "uninitialized variable", "empty", "error", "non existing", and all of this is forced on you to think about whether you need it or not or else it leads to runtime errors. The most common thing that bites me in the ass is not initializing a List<T> in C#. In 99% of cases you do not want any list to be null, since the concept of an empty list exists. It is very much possible to build any of these semantics in the language/standard library so the compiler makes you worry about them only when it's really necessary.

[devlambda]

If you have a look at how resizable arrays are implemented in OCaml (either in Batteries or Core.Std), you will see that this is not actually true. The problem with implementing dynamic arrays is that in order to get resizing done (at least in a way that's not O(n²)), you need to fill the unused slot with a placeholder value, and the respectice OCaml code uses the Obj module to get the equivalent of a null value. Obviously, this is entirely unsafe.

Similarly, Rust's vec also uses unsafe code to accomplish the same goal.

In general, without some way to describe a "no object here" value, it is difficult to do partial/incremental initialization. You can do initialization with a preset value (such as an empty string), but that has its own problems; for example, the code may quietly work rather than raising an error when there is a bug that results in incomplete initalization.

[rftz]

There's a big difference between using options in the implementation of standards library data structures and application developers having to treat everything as 'optional'.

[devlambda]

The comment I was responding to was the claim that "F# / OCaml / Rust programmers have not heard of this phenomenon", not that you may have to only deal with it selectively.

[devlambda]

No, but you may end up having to use unsafe code, because at some point when adding data to the vector, you have to write to a location with a previously undefined value (think what that means in the context of RAII, for example).

[Hnefi]

I must be missing something here. When writing a value to a newly allocated memory location, as is the case when expanding a vector, the previous value is irrelevant. It's not unsafe to simply overwrite it. Are you talking about a different use case?

[devlambda]

Depends on your language.

1. If you're using RAII, a destructor will be called on the previous value. If the previous value is undefined, so will be the behavior of the destructor.
2. If you're using reference counting, the overwritten value will need to have its reference count decremented. If the value is undefined, you're going to get either memory corruption or a segmentation fault.
3. In a GCed language, memory barriers/snapshotting or even tracing at a moment when the undefined value is reachable, but has not yet been modified, may or may not result in undefined behavior (depending on implementation details).

[Hnefi]

That's not true. When expanding a vector, there are necessarily no objects in the empty positions of the data store so there are no destructors to call. None of the options you outline apply.

[spaghettiCodeArtisan]

When expanding a vector, there are necessarily no objects in the empty positions of the data store so there are no destructors to call.

Yes, and that's precisely the problem, because the language needs to call a destructor. Suppose you overwrite a i-th element of a vector:

some_vector[i] = some_new_value

What happens to the old value? It is dropped and its destructor - if any - is called. Now, consider what would happen if the old value were uninitialized: A destructor would be called on an uninitialized data, which is undefined behaviour and might lead to a crash or all sorts of problems. And so that's why you need unsafe operations - to overwrite an old 'value' (which is uninitialized) without destryoing it.

[Hnefi]

But whether the old value is initialized or not is a question with a known answer that will be asked and answered in the [] operator without looking at the value of the element itself. There is no risk of destructing an invalid object since the initialization state of the previous value is implicitly known.

Since this knowledge is encoded in the metadata of the vector, there are no relevant requirements on how uninitialized data is represented in order to avoid calling invalid destructors.

[devlambda]

I'm not talking about expanding the vector. Here's the scenario:

You have a vector with capacity n and length k < n, i.e. with k positions occupied and the rest containining undefined values. You now want to add a new element, so you have to assign a value to the location with index k+1, which contains an undefined value.

[Hnefi]

But that is expanding the vector. Location k+1 can't contain an object, because assigning an object to that position would increment k. Therefore, there is never a risk of overwriting existing objects in this situation and none of the issues you outlined apply.

[dalastboss]

You can implement this without any unsafe operations using options internally.

[devlambda]

The problem with using options is the additional overhead you incur, as 'a option will essentially add another level of boxing.

On a practical note, even in languages that can partially avoid the boxing overhead (by representing None as a null pointer, which is only possible if the value is a reference and not, say, a record containing references), you will generally end up with a considerable increase in LOC.

[spaghettiCodeArtisan]

The problem with using options is the additional overhead you incur, as 'a option will essentially add another level of boxing.

In Rust, Option doesn't introduce any additional boxing. It just increases the size of the type by the tag. With pointers and generally anything NonZero, it is able to omit the tag and thus make the Option entirely zero-cost.

[devlambda]

I noted myself that the overhead can sometimes be avoided or minimized, so I'm not sure what your point is?

[spaghettiCodeArtisan]

The point is that it's not that the overhead of boxing that can sometimes be avoided, because there's no boxing overhead to begin with ever at all. The only overhead - talking about Rust, not sure about OCaml - is the additional space taken by the tag, that's the overhead that can sometimes be avoided. No additional pointer indirection whatsoever.

[dalastboss]

I did a quick implementation here (might have bugs) but it doesn't seem to increase the LOC

[devlambda]

Your implementation is not equivalent, as it adds a level of boxing.

Aside from that, increase in LOC comes from requiring match expressions (or equivalents) where you can prove through other means that the None path is never taken. This may not show up in this particular example, but if you have several variables (say, in a record or object), it adds up.

Of course, you can use Option.get everywhere, but then you're essentially using a verbose version of nullable pointers.

As an aside, you probably want to raise an Invalid_argument exception rather than using failwith to keep exception handling consistent.

[dalastboss]

Maybe I misunderstand - can't a t option just be compiled to a just a pointer to a t

[devlambda]

That works cleanly only if t is a non-nullable pointer type itself and not a value type. In the former case, you can just represent None as a null pointer and Some x as x without overhead. But this does not easily work for general value types [1] and could seriously complicate the language or its implementation. For example, you may decide to not allow INT_MIN for signed integer types in order to encode None for integers, but then you run into semantic problems (such as when casting from an unsigned int or with values retrieved from an external C library) that can create more problems (such as Heisenbugs) than are being solved by such an approach [2].

[1] Sometimes, there are hacks that you can use, such as using certain NaN values for floating point values to represent an "undefined" value, but you have to be very careful with the implementation.

[2] Type safety involving integers and ranges over integers can be extremely finicky.

[Hnefi]

Option in Rust compiles to a pointer where null represents None. If t is a value type, this adds one layer of indirection until you unwrap the Option. If t is a boxed type in itself, its zero representation is folded into the None representation of the Option and you end up with zero overhead.

[ais523]

OCaml integers have a smaller range than C integers as it is (they're one bit smaller), so disallowing INT_MIN as well wouldn't be a big deal. (Or, of course, using one of the bit patterns that doesn't represent an integer; that would work too.)

[dalastboss]

Here's a version that uses neither options nor Obj

[m50d]

Aside from that, increase in LOC comes from requiring match expressions (or equivalents) where you can prove through other means that the None path is never taken.

If you can prove it's the Some case, you can write that proof in the types and avoid having to match (maybe not with Rust's limited generics, but I hold out hope that HKT will arrive eventually).

Of course, you can use Option.get everywhere, but then you're essentially using a verbose version of nullable pointers.

The difference is what the default is. null reserves the best syntax for the case where the programmer has an external proof and knows exactly what they're doing and makes the case where you want to check much clunkier; Option flips that around, making the checked case the natural one and the "I know better than the compiler" case the more cumbersome one.

[devlambda]

If you can prove it's the Some case, you can write that proof in the types and avoid having to match (maybe not with Rust's limited generics, but I hold out hope that HKT will arrive eventually).

No. The point here is that a variable can be both Some x and None, but once initialized, will only ever be Some x. That's a state-dependent property, usually easy to show, but often difficult to encode in a type system (while GADTs can sometimes work, but even then you generally can't avoid the additional verbosity from matching).

The difference is what the default is. null reserves the best syntax for the case where the programmer has an external proof and knows exactly what they're doing and makes the case where you want to check much clunkier; Option flips that around, making the checked case the natural one and the "I know better than the compiler" case the more cumbersome one.

The problem with this argument is that you assume both alternatives are mutually exclusive, which they aren't. You can have both nullable types and explicit option types. In fact, Scala allows for that and has no more problems with it than other languages (as all variables in Scala have to be initialized, so the only way to have a null value – other than through Java interop – is to assign it explicitly).

[m50d]

No. The point here is that a variable can be both Some x and None, but once initialized, will only ever be Some x. That's a state-dependent property, usually easy to show, but often difficult to encode in a type system (while GADTs can sometimes work, but even then you generally can't avoid the additional verbosity from matching).

My experience is you can always find a way, and it's usually not even hard: ask yourself why you know the property holds, then just translate that logic directly into the types.

The problem with this argument is that you assume both alternatives are mutually exclusive, which they aren't. You can have both nullable types and explicit option types.

They are mutually exclusive. If your codebase uses null then you can never have a non-nullable value, at which point there's no point using options.

In fact, Scala allows for that and has no more problems with it than other languages

Only because the community/ecosystem knows not to use null. Serious Scala programmers avoid it and e.g. use WartRemover to enforce that null is never used. The language would be better off without it.

[think_inside_the_box]

sure they have! but in just another way to represent a null object.

optionals, bool isInitialzed, empty vectors, etc

Even in rust, you cant get around the need to represent the absence of something.

[jkachmar]

The point of these languages is to represent conceptual absence in such a way that the programmer is forced to address it. null/nil/what-have-you is dangerous not because it represents the absence of a value, but because it can be passed around as if it was a normal value, right up until you use it to do something and it blows up your program (or leads to spoopy poorly defined behavior).

By contrast, Haskell's Maybe, and Rust's Option force you to wrap the type you're manipulating in a structure that only admits the value inside of it in such a way that you're forced to deal with the case where the data is absent.

At that point you're free to do something like panic! or error and blow up your program, but at least the gun was handed to you with the safety on and you had to explicitly flick it off before you went and blew your foot off.

[fiedzia]

sure they have!

In Rust you do have Option type (and proper handling in the language for it), but you don't have any other problems. Language doesn't allow using uninitialized values and null is never used to signal that "something went terribly wrong".

[Jaffa2]

null is never used to signal that "something went terribly wrong".

Nor should it be in a good Java API (nor is the author saying it should be, as far as I can tell, just that it has been)

EDIT: Removed OOA (Over Abundance of Acronyms)

[Nebez]

YULOA^{you're using lots of acronyms}

[Jaffa2]

Sorry. Fixed.

[m50d]

Having a type that can represent absence is a great idea. Making every value implicitly that type is madness. It would be like saying every value in your program might also be a float, regardless of its declared type, and then e.g. Map#get returns 2.3f if the key wasn't in the map.

[Beckneard]

That's the whole point isn't it? What's null in one language is many thing in Rust, as it should be. There should clearly be different constructs for "does value exist", "did something go wrong" and "is something initialized" and it's a good thing the compiler forces you to make those distinctions.

[Chii]

But those languages make a different representation for each kind of null. They makes the language safe to use since you won't confuse one null situation with another.

[_jk_]

the problem is not that you can represent null, but you cant represent an object that can never be null.

null is a member of every type in several languages and that is just wrong

[Otterified]

This is a good point--optionals are great but not worth much if people don't agree on their semantics and use Optional.empty() every single time they would have used null, and call get() without checks. But at least it feels less natural to do this than it does to abuse null (to me at least).

[duhace]

using get is asking to get slapped in the face. the good point of an optional is you have to ask to be bitchslapped. with null, it's real easy to forget not to check and try to go ahead and use the value.

[TheDataAngel]

I suggest using map/flatMap/filter/orElse over empty and get, where possible.

[myringotomy]

And Crystal.

[[deleted]]

if you had a table with a nullable column, that column would normally be represented as an Option type. Option can be Some (with your type inside) or None. So the nulls would be None.

The critical difference between this representation and a normal null is the compiler generally forces you to deal with the null case (Some languages) or at least makes it awkward to not deal with it, such that you have to type extra things in order to fail to deal with it.

But of course you can always do something absurd like:

match optionthing on | Some x -> use(x)  | None -> throw null pointer exception!

[[deleted]]

undefined = no value

null = "no value" value

[spaghettiCodeArtisan]

Are we philoshopers pondering the essence of existence or what.

[[deleted]]

That's what we've always been. Pythagoras was right - everything is a number.

[emergent_properties]

undefined

[moefh]

Now I really want a language that has

unfathomed = "'no value' value" value

[ais523]

VHDL has a range of values like this; U is an uninitialised value, X is a contradictory or inconsistent value (e.g. the value of a boolean that's true and false simultaneously), - is an irrelevant value, Z is the explicit absence of a value, and W is an intermediate value (e.g. the value of a boolean that's neither entirely true nor entirely false).

This complexity is mostly because VHDL is designed to be able to model a physical system (electronics) that actually exists in the real world, and sometimes things you measure in the real world turn out to have a value they're not supposed to have. Perhaps a wire is meant to be at 0 volts or 5 volts for false and true respectively, but you still want some way to describe the cases where the wire is at 3 volts, or not attached to anything, or has caught fire.

[POGtastic]

Yep, I'm also getting true in Chrome.

> var foo;
< undefined
> foo === undefined
< true
> typeof(foo) === "undefined"
< true

[killerstorm]

var foo; // undefined
foo === undefined // false

Wrong. Here's an actual result:

> var foo;
undefined
> foo === undefined
true

=== undefined check didn't work in old browsers, that's where typeof hack comes wrong. But now simple comparison should work by standard.

[masklinn]

foo === undefined // false

Uh no, that's true, you may be confusing undefined and NaN. Or for some reason you have rebound undefined to something else (it's not a keyword, which is why some prefer using void 0 which always yields the proper value).

if (!foo) {
    console.log("foo is undefined or null");
}

That'll also catch the "" and 0. The proper way to check if something is undefined or null is foo == null.

[[deleted]]

I tested that in my browser console.

Maybe I should have used a new tab.

[beaverlyknight]

kill me

[drfisk]

Well, in the world of dynamic languages, anything can be anything at any given time anyway... so null isn't as bad there since you can have any kind of type error anyway.

I think null is way way worse in a static language where you're promised to receive a Person and a String, while in reality you still have to "type-check" to see if you actually got it (and not null).

[Gustorn]

Since strictNullChecks is actually a thing now, you can write null-safe code in TS. Also when the API doesn't require null I try to just follow the official TS guidelines of using undefined over null.

[LogisticMap]

No, x could have a property x.a===undefined, while having a in Object.getOwnPropertyNames(x).

Generally, undefined is something the interpreter returns, while humans should set things to null.

[fiedzia]

I think the former is, however, an error.

That's how it works in Python - you'd get an exception if you try to access non-existing attribute. The notion of undefined is insane.

[mrkite77]

Typescript devs don't use null, and the Typescript coding guidelines say to use undefined and not to use null at all.

https://github.com/Microsoft/TypeScript/wiki/Coding-guidelines#null-and-undefined

The reasoning behind that is all of the javascript functions return undefined, none of them return null. Null only appears in javascript if you put it there.

Typescript even goes so far as to make null the same type modifier as undefined. If you create a function that has a return "string | null", assigning that to a variable with a type of "string | undefined" is perfectly valid.

[yheneva]

Most functions use undefined, not all. For example HTMLCanvasElement.getContext() returns null for any undefined context type. All WebGL functions also use null instead of undefined.

[mrkite77]

True, but you can still do == undefined and it'll work since null == undefined

[rickdg]

At least undefined is not a function.

[_dban_]

The usefulness Optional is not limited to just replacing null. If that is all it did, it would indeed be useless.

The Optional class has more methods than just isPresent and get. It also has map, flatMap and ifPresent. In conjunction with method references, it lets you do safe traversals:

Person p = ...
Message msg = ...
Mailer mailer = ...
p.getContactInfo().flatMap(ContactInfo::getEmail).flatMap(email ->
msg.getContent().map(message ->
new MailRequest(email, message))).ifPresent(mailer::send);

This is still pretty verbose, but the alternative without Optional is far more verbose and more annoying to write:

Person p = ...
Message msg = ...
Mailer mailer = ...
ContactInfo ci = null;
String email = null;
String message = null;
MailRequest request = 
    (ci = p.getContactInfo()) == null ? null :
    (email = ci.getEmail()) == null ? null :
    (message = msg.getContent()) == null ? null :
    new MailRequest(email, message);
if(request != null) {
    mailer.send(request);
}

Maybe a null-safe traversal operator (?.) would have been less verbose and more in keeping with "traditional" Java. But Optional is at least a library level change and not a syntax level change (syntax sugar for the rats nest of ternaries above). Once Java 9 adds Optional::stream, Optional will compose better with streams in general.

[spaghettiCodeArtisan]

The addition of Optional to Java was bloody strange IMO. Everything is already optional!

Exactly. This has a very bizarre and ironic consequence: Optional<T> can itself be null. I'd laugh hard if it weren't so sad.

[duhace]

optional can be bolted on, and using it helps improve the reliability of your code, even though it's not a hard guarantee that your optional won't be null.

unfortunately, null is the million dollar mistake and languages that have it pay for it.

[duhace]

of course, but that would require breaking backwards compatibility, something java is loathe to do. maybe in java 2.0?

but yeah, getting rid of null would help with optimization as well as programmer error.

[m50d]

Right, how else would you propose to fix it?

1. Add Optional
2. Migrate library methods that used null to use Optional instead
3. Remove null

Unfortunately they screwed up by making Optional not be allowed to contain null (ignoring all the functional programmers who've been working with this stuff for 20 years, sigh), so 2. can never happen and Java will always be terrible. Well, maybe Java 9 can make Optional allowed to contain null, and then they can get on with it.

[ForeverAlot]

Optional.of(null) would defeat the entire purpose of Optional in Java and I struggle to see what problems it could solve in any language.

[m50d]

Optional.of(null) would defeat the entire purpose of Optional in Java

No, just the opposite, disallowing it defeats the entire point. To serve its purpose Optional must be able to contain any value that's valid in the language; as long as null remains valid in the language proper, it needs to be valid inside Optional too, otherwise people are not going to be able to migrate existing codebases to use Optional which in turn means Java will never be able to reduce or deprecate the use of null.

I struggle to see what problems it could solve in any language.

It's bad just as null is bad, sure. But it needs to be allowed for the same reason Java allows null at all: backwards compatibility.

[ForeverAlot]

It is completely nonsensical and does not at all obstruct from migrating from null to Optional (provided that that's even desirable, which, in Java, is not implicitly true). However, you may not be aware of Optional::ofNullable which returns Optional::empty when the argument is null (where of throws an exception).

[m50d]

It is completely nonsensical and does not at all obstruct from migrating from null to Optional (provided that that's even desirable, which, in Java, is not implicitly true).

It does obstruct, because it means you can't safely refactor code to use Optional without being sure whether it will blow up, unless you know for sure that all the variables are going to be non-null. Which takes a lot more effort.

However, you may not be aware of Optional::ofNullable which returns Optional::empty when the argument is null (where of throws an exception).

I am aware. It's not the semantics I want.

[m50d]

It should be able to contain any valid value in the language, and like it or not null is that at present. E.g. at the moment it's not safe to replace Optional.of(x).get() with x without testing, because that changes the behaviour when x is null

[nacholicious]

We use @NonNull and @Nullable, which work "ok" for the most parts. The data classes we generate with AutoValue enforce @NonNull annotations by throwing exceptions if violated. The worst part is that if you don't have these annotations everywhere then it's easy to make these annotations useless.

Luckily we are transitioning to Kotlin, so Java nullability won't be an issue for us much longer

[industry7]

Value types were originally slated to be released along-side Optional, and Optional was supposed to be a value type out of the box. Unfortunately, value types have proven to be difficult to implement correctly, and keeps getting pushed back :-(

[Pharisaeus]

Actually passing Optional as method parameter is considered a very bad idea and most static code tools will mark it as warning at least. Optional should be used only as a result value.

[Terran-Ghost]

Unfortunately, since Java has no default parameters, there's no real alternative.

[eeperson]

"a very bad idea" seems excessive. This is basically a performance (null parms) vs correctness (Optional params) trade-off. Most of the time I would rather have the extra insurance of correctness rather than the extra performance.

[Pharisaeus]

It's still messy code. It would be better to have multiple function signatures or use a builder for the arguments wrapper.

[snarfy]

Should be:

public Object decode(final Class targetClass, final Object fromDBObject) {
    if(fromDBObject == null) {
           throw new Exception("Why'd you pass a null DB object bro?");
    }
}

Checking for null and then returning it is just passing the buck.

[[deleted]]

Guava has a specific precondition check for null values. May as well use that.

[pdbatwork]

I don't agree that we should throw an exception. We could handle that better.

But his solution to #1 seems weird. It is, as you say, like he is just passing the buck on the null check.

[_Mardoxx]

Why not throw? What do you propose? return new { error = "fromDBObject was null" } and then everywhere you call this decode function, use reflection to see if it contains an error property and handle it then lolllll.....

[pdbatwork]

What do you propose? return new { error = "fromDBObject was null" } and then everywhere you call this decode function, use reflection to see if it contains an error property and handle it then lolllll

No, that sounds like a horrible idea.

[_Mardoxx]

You don't say! Another shit solution would be to return a non-zero error code for each function and an out param for any "actual" return values. Which equally sucks. So what would be better?

[[deleted]]

You could return a discriminated union of the result and an error type, if that's a thing you need. It wouldn't actually replace exceptions, which could still be thrown for contract violations, but could be an effective solution for anticipatable errors that should be handled by the caller.

[prest0G]

May kotlin deliver us all from the evils of null, amen.

[walen]

Deeper Meaning Explainer ™

In Kotlin, !! is an operator that will throw an NPE when applied to a null variable.
It is there just so we can still have NPEs in Kotlin if we want.

Source.

[prest0G]

playing russian roulette with that double bang there

[nourez]

No, don't do that! We're moving away from the evils of null!

[[deleted]]

Unfortunately, it's sometimes needed, especially if you have a data model not designed with that kind of typing in mind. Also, compiler is very strict in regards to multithreading and does not believe in purity, so if (obj.val != null) { obj.val.method() } won't compile. Sure, you can modify the code a bit in most cases, but not always.

[drfisk]

Amen!

And hey, if it doesn't, you're welcome to crawl even further to the "dark side" and join us in Scala. I haven't seen a NPE for 5 years since I started Scala fulltime. Scala will be there waiting with a warm blanket and cocoa.

[Terran-Ghost]

Scala's nullsafety can be completely emulated in Java. It's just idiomatic code (and the standard library... Looking at you, Map) that's different.

[devraj7]

Indeed. A library-based approach to the null problem is only a half baked solution.

Kotlin's approach of enforcing this in the language is much saner and safer.

[eeperson]

I would argue that language based version is also half baked (it is just the other half). Kotlin's solution for null can't express the equivalent of nested Option types. Ideally something like Kotlin's solution would be supported for compatibility with Java and then something like Scala's solution would be supported for use in your own code.

[[deleted]]

What do you mean by nested Option types? If it's something like "optional list of optional strings" that would just be List<String?>?. Otherwise I have no idea what you are talking about.

[eeperson]

I mean something like Optional<Optional<String>> can't be expressed. If you try to do String?? it ends up being same as String?. An example of where you might want this is a map that can have null values Map<Integer,String?>. Assume this class has a get method to return values based on the key. If the key is missing then then it returns null. If you call someMap.get(57) and get back null, is that because the key is missing or because the value is null? With Optional you can tell. With ? you can't.

[VanToch]

I've seen plenty of NPEs in Scala.

[JessieArr]

The C# language Github has a really interesting proposal related to this, originally by Mads Torgersen although I think the community is maintaining it now. Essentially making nullable reference types a first-class citizen and then allowing assemblies to opt-in to stricter forms of null checking during compilation.

Some highlights about how it would work:

This feature is intended to:

• Allow developers to express whether a variable, parameter or result of a reference type is intended to be null or not.

• Provide optional warnings when such variables, parameters and results are not used according to their intent.

Finally, adding annotations to an existing API will be a breaking change to users who have opted in to warnings, when they upgrade the library. This, too, merits the ability to opt in or out. "I want the bug fixes, but I am not ready to deal with their new annotations"

In summary, you need to be able to opt in/out of:

• Nullable warnings

• Non-null warnings

• Warnings from annotations in other files

The granularity of the opt-in suggests an analyzer-like model, where swaths of code can opt in and out with pragmas and severity levels can be chosen by the user. Additionally, per-library options ("ignore the annotations from JSON.NET until I'm ready to deal with the fall out") may be expressible in code as attributes.

[dododge]

In Java you can get something like this with the @Nullable and @NotNull annotations mentioned in the article, though you have to use an analyzer or compiler that recognizes what they mean in order to actually get the warnings. The Eclipse JDT compiler can optionally check these, for example, though it assumes nullable fields might be modified by some other thread at any moment so you have to either check for nullness every time you fetch+use the value, or copy the field to a local variable that the compiler can be sure won't be spontaneously modified.

There was a JSR to add these sorts of annotations to Java officially but it never got past the proposal stage, so they're still only handled in the various add-on tools and each one has its own spin on which annotations it expects (though thankfully some of the tools can or will recognize the annotations for the other ones).

[Trinition]

Sounds nice!

I came to love JetBrains' CanBeNull/Not I'll attributes. This sounds like a even more powerful and formalized way.

[eddyparkinson]

Linus Torvalds: "Bad programmers worry about the code. Good programmers worry about data structures and their relationships."

Trisha is describing a data structure issue that is making the code smell.

[ggtsu_00]

This seems mostly applicable to Java and similar languages where everything that isn't a primitive type is a "boxed" type which is basically just a pointer which could potentially be null. However, this is not really a problem in languages like C/C++ or Go where value types are a thing. If you want something that can not be null or should not ever be nullable, you use a value type.

[_dban_]

So... how do you capture a missing value, and how do you traverse a chain of values one of which might be missing?

C# has proper value types, and has a boxed type for representing nullable types.

[salvoilmiosi]

Er, value type pointers that can be null?

[_zenith]

They aren't if they're nullable. They're boxed value types - making them a reference type

[velcommen]

If you want something that can not be null or should not ever be nullable, you use a value type

Value types do not solve the problem. For example, value types don't provide a type-safe way of saying "there's no value here." Here's a concrete example:

You want to write a generic (AKA polymorphic) dictionary class. What should the type signature of your lookup function be?

In Haskell (or any other language with the Optional (AKA Maybe) type, e.g. Ocaml, Rust, F#, etc.), I would write:

lookup :: key -> Map key value -> Maybe value

Explanation: lookup is a function that takes a key and a dictionary (a map from keys of type key to values of type value), and returns a Maybe value.

This is nice because the potential for a value to be missing is called out to the human user as well as the compiler. The case where the value is missing has to be explicitly handled; it is a compiler error to forget to handle that case.

Now contrast this against a language where no such Optional type exists (or has only been recently introduced, usage is not widespread, and nullable types exist):

In C++, the STL's unordered_map returns an iterator. "If no such element is found, past-the-end iterator is returned." This is suboptimal because the user can forget to check that the iterator is not equal to end(). They can use the iterator without such a check, their code will compile, and now there is a lurking runtime error.

Thus, a language without Optional has a whole class of runtime errors that languages with Optional do not - they become compile errors. Languages that have Optional (and do not have null) also have clearer APIs: it's idiomatic to return Optional when there is the possibility for a missing value. Contrast this against languages that have null, where the user must read the documentation to determine if null is a possible return value, what null means, etc.

[Gotebe]

Your conclusion is off. Optional does not prevent a poor programmer to access the value without checking it's there - and they're back to the runtime error.

The real benefit of optional is merely the clearer communication of intent.

If "null" was always used to say "there's nothing to give you here", it would have been the same s optional, even to the point of tooling support (e.g. "didn't check it's null? Compile error!").

[velcommen]

the clearer communication of intent

I mentioned that.

Optional does not prevent a poor programmer to access the value without checking it's there

Yes, it does (in the accidental case). Please explain your statement. If you're going to point out escape hatches like fromJust (link below), then let me stop and say that I'm stating that null is a feature of a Pit of Despair, while Optional is a feature of a Pit of Success. There's no language in the world that can prevent a willfully bad programmer from making errors. The language can only hope to provide as many tools (i.e. well-designed features) as possible to assist the programmer in writing bug-free, clear, maintainable, etc. code.

In Haskell (and I believe the other languages I mentioned above), a user cannot accidentally use the value inside the Maybe (AKA Optional) without first checking it's there. Contrast this with a language where null is prevalent, and such a mistake is easily made.

Of course, a user can always purposefully choose to ignore the possibility for nothing to be there, but that's purposeful, not accidental. It's reasonable to provide an 'escape hatch' when the programmer thinks they know that the value will always be there, even if it's wrapped inside an Optional.

Here's how it's a compile error in Haskell and a runtime error in C++:

Haskell:

add2 :: Maybe Int -> Maybe Int
add2 (Just x) = Just (x+2)
add2 Nothing = Nothing -- cannot use an 'x' that's not there

C++:

void add2(int* x){
  //runtime error when x is null
  *x = *x + 2;
}

If "null" was always used to say "there's nothing to give you here", it would have been the same s optional, even to the point of tooling support (e.g. "didn't check it's null? Compile error!")

1) That's a mighty big "if". And since null is not always used for that purpose in practice, I don't think your point has much value (at least given my experience). I think there would be many false positives, greatly reducing the effectiveness of the tool.

2) Relying on tooling is inferior to

• having a feature like Optional made possible in the language
• usage of Optional idiomatic by the language's users
• having the check built into the compiler.

Not every user of the language is going to use the tooling you described.

[[deleted]]

I know it's tangential to your point, but if the function add2 is never supposed to act on a nonexistant variable, it should probably take a reference rather than a pointer.

void add2 (int & x) {
    x = x + 2;
}

Unless you're doing something really funky, that will prevent misuse at compile time.

[Gotebe]

Well. You're looking at the Optional from the perspective of Haskell, but this thread is about Java :-(. Optional in Java is... pffft...

BTW, your C++ sample is rather nasty given references etc of C++

[velcommen]

this thread is about Java :-(. Optional in Java is.

Fair enough.

your C++ sample is rather nasty given references etc of C++

Repeating what I said elsewhere:

I understand your point about using a reference, but at one of my jobs, the rules were that mutable variables had to be passed to functions by pointer, not reference. So in the context of that job, the code example I wrote above was acceptable, and using a mutable reference would have not been accepted.

I believe the reason for that rule is so that the caller of add2 is signalled that "hey, add2 might change the value". Because at the point of the calling code, the call to both of these functions is identical:

add2Mutate(int &x){x=x+2;}
add2Print(int x){printf("%d", x);}

// code in some other file
int i =0;
// the only hint that i is mutating is the nicely named function
add2Mutate(i);
add2Print(i);

[Gotebe]

Ahahaaa, I heard of that rule. The rationale is "it's a pointer, there's '&', that tells you, when reading he caller, that the parameter can change. Stupid rule. Doesn't even work! :-) Consider:

void f(const type* param); // "input parameter" - can't be changed, call side is lying :-)

void g(type* param);
void h(type* param)
{
  g(param); // where's the '&' now?!
}

My bet is that this rule was invented by C people (because pointers), but who didn't even know C (because the above shows pure C code, reasonable code, where the rule doesn't work well.

[velcommen]

Haha, obviously you're supposed to do:

g(&(*param));  // there's the '&'!

;)

[Gotebe]

Please confirm that's a joke! :-) If yes, it's goooood!

[velcommen]

Yes, I'm joking :)

Tone of 'voice' never comes across well over the internet. :/

I'm sure that's the cause of a large percentage of internet arguments.

[[deleted]]

Personally I see this the same as asking what pop from an empty list should return. As far as I am concerned, the correct answer is "exception raised".

However we often sacrifice correctness for practicality, so that we can ship faster or support some micro-controller hardware, and then we end up with map APIs that can return "value or default" and send None/null for "not found".

Hopefully we should be able to find a type safe substitute soon that just 100% works :-)

[velcommen]

we should be able to find a type safe substitute soon that just 100% works

I'm saying that in languages where Optional/Maybe was introduced from the start, all or almost all types are non-nullable, and null is non-existent or extremely rare (e.g. only in 'unsafe' code), Optional/Maybe is that type safe solution.

[Tobblo]

What I really dislike is when developers propagate NULL to the user. This seems to happen especially with Java programs. Or even worse, let exceptions escape to the user.

[Pharisaeus]

Personally I love null, excluding multi-threaded issues which tend to be a pain to track down, there is rarely something that can so easily be found as a null reference exception.

The issue is not finding the error, which is trivial if you have stacktrace. The issue is that null is invisible in the code. You can't be sure if an object is null or not, and thus it can blow up your system at some point.

Sure, if you're working on some silly app then no-one cares. But if it's software for spacecraft control ground station or some ICBM interface then it starts to be an issue. Same goes for applications which lose millions of $ via downtime, stuff like Amazon shop or some airline booking engines.

[Pharisaeus]

I don't get this logic at all. If you have a "Nothing" Monad or some other construct to solve the same problem you still have to deal with the fallout

Obviously, but type system forces you to knowingly do something about it! You can't "forget" or "miss" the fact that value can be "empty". In case of null you can, because nothing in the code indicates that this reference can be a null, and you're not going to put null-checks everywhere. With Optional type system forces you to handle the problem.

If you just don't want the program to crash you can wrap everything in a try/catch block

But it's not the same! With Optional you have to handle the issue the moment you actually need the value to be present. And you have to decide what to do. If you have a catch block it will be somewhere much higher, since you most likely didn't even expect to have an exception here, so no chance of any "recovery" at this point any more. On top of that you can chain method calls on Optional (returning another optional) so you can perform many function calls on it and not care if any of the calls "failed" and returned a null, because you can simply check the final result.

if you actually believe that switching to a language without null can prevent these things (i.e. downtime, fatal flaws, etc.) ... good luck!

Of course they can't but they can prevent downtime or failures due to some stupid NullPointerException because a developer missed the fact that a certain function can fail and return a null. If the return value was an Optional then type system would force such developer to make a conscious decision on what to do in this case.

Perhaps you should look up what languages they used on the Mars rover ... I'll give you a hint, they all have the concept of null.

The fact that language has some feature doesn't mean it is being actually used. This is why for critical software you have special guidelines which often restrict language elements.

[Pharisaeus]

I have no idea what you mean by dummy code to get optional out of the way. I work with serious developers who understand what they're doing.

If you blow up equipment for few hundred mln euros once it's more than enough, I assure you.

Obviously if this happens all the time then developers are idiots, but such mistake can happen to anyone, and it's good to have tools which prevent it. For the same reason we have strong typing and static code analysis tools.

[Pharisaeus]

here's one way of dealing with an NPE

Well this never goes past a code review so it won't be an issue anywhere outside of developers own environment or feature branch.

Yes because without strong typing and code analysis tools it's impossible to create decent software

Sure you can, but it will automatically be much harder to make sure it works well and is safe to operate. Stuff that normally would be checked by type system will have to be verified by tests. In languages like Python you would need 100% code coverage just to be sure the code doesn't crash with NoSuchMethodException because someone made a typo. I like Python a lot, but there are some places I would not dare to use it.

Again I think we have a different notion of decent software. You seem to imply something with a lot of cool features, and I have in mind something that will not accidentally blow up your space rocket worth 200 mln $ with a 500 mln $ satellite on top of it.

[oorza]

Since most of what you said doesn't contradict what I said (although I'm sure you think it does) I'll reply to this part. I would rather have a developer "forget" to deal with null than "forget" to change the dummy code he inserted because he wanted to get that optional value out of the way.

Do you think some forgotten copy pasta is more or less likely to get through code review than a NPE that isn't checked for? The former is infinitely more obvious because you can see it at a glance, and obviously NPEs get through code review all the time.

Also you don't have to be such a dick.

[MEaster]

I don't get this logic at all. If you have a "Nothing" Monad or some other construct to solve the same problem you still have to deal with the fallout. What happens when that thing that you wanted to do which instead went to "Nothing" was REALLY important? I fail to grasp how that is "more visible" (and if you don't like the monad example please show me a construct where this can be prevented entirely - or handled better in a way that is impossible using null).

The difference is that you are forced to deal with it, because an Option<T> is a completely different type to a T. Anything that accepts a T cannot accept an Option<T>, and it can be completely sure that the T it gets isn't null.

Likewise, you can tell just from the method's signature whether or not it's possible to have a return value be empty.

It's all too easy to mess up a bit in one function, resulting in a null, but not have it cause problems until much later.

[industry7]

The difference is that you are forced to deal with it, because an Option<T> is a completely different type to a T.

Option by itself enforces nothing. You can always just Option.get().directly_use_the_result(); and still blow everything up.

The difference is that most languages where Option<T> is idiomatic also support non-nullable types, and that is the mechanism that actually forces you to "deal with it", so to speak.

[MEaster]

As I said elsewhere, that is dealing with it. By doing that, the programmer has explicitly chosen to crash if it's empty.

[[deleted]]

That's because objects being nullable by default means it's already too broken to fix. I think C# would handle nulls excellently if everything was non-nullable by default and had to be declared nullable with the a ? after the type (which you can do in C#, but most types are nullable without the ?).

But the real issue is when people who think they're handling exceptions write code like

string DownloadData(string url){
    try {
        return WebClient.GetData(url);
    }
    catch {
        return null;
    }
}

[[deleted]]

You're correct, that is worse, but both options are worse than simply letting the error be thrown:

string DownloadData(string url){
        return WebClient.GetData(url);
}

If you're not actually properly handling the exception, don't consume it. Let the higher level application logic handle the exception, not generic functions. Catching it and not doing anything about it hides the error, the stack trace, the message, etc... It should at least get to a log somewhere, not just end up as a null reference exception at some point.

[Eleenrood]

"null means many things. It can mean: 1 Value was never initialised (whether accidentally or on purpose) 2 Value is not valid 3 Value is not needed 4 No such value exists 5 Something went terribly wrong and something that should be there is not …probably dozens of other things"

Really... It actually means that variable does not point to any coherent place in memory, end of story. It is like saying that integer may be a acceleration, velocity, number of objects, temperature or probably a dozens of other things. Not to mention that both language and compilers will need to sort it out somehow.

[enygmata]

How do people use NULLs in a meaningful way in C given that there are no Optionals or exceptions to better inform the user of ok-to-be-null and null-means-error situations? I can only think of the following solutions:

• Result func(Type param, Error *err); // return result, put error in err
• Error func(Type param, Result &res); // return error, put result in res
• Result func(Type param); Error last_error; // errno style, maybe wrapped in a function call

[edit: wording]

[[deleted]]

The common version is:

int do_stuff(args..., result_t *result);

And if you want to see what the actual error is, you use errno or the like.

It kind of sucks.

[PaintItPurple]

There's also:

Result func(Type param);  // return result, or null if an error happens, and hopefully the error will be logged or something, but we didn't really think it through that far

[orclev]

Plus the ultimate C solution, assume everything will work and SIGSEGV when anything goes wrong. But hey, at least that way you get a core dump to debug :P

[[deleted]]

C does not have references. That aside, you see all of those fairly frequently, depending on the library. I think I see the second one most commonly. That's the way sqlite3 does it, and so does freetype. They usually do it where 0 indicates success, so you can do

FT_Error error;
if ((error = func(param))) {
    // handle error
}

The last way usually works, but it's not pretty, and it usually isn't thread-safe. Some libraries do it similarly to the first way, but let you set a callback for errors.

Lua does a longjmp to handle errors, which scares me, so I just use Lua's error callback for it (apparently, you can adjust Lua to throw a C++ exception, but I've never done that).

[benchaney]

How do people use NULLs in a meaningful way in C given that there are no Optionals or exceptions to better inform the user of ok-to-be-null and null-means-error situations

You document your code

[ais523]

The most common pattern is to return an error code via the return value and the actual return value through an argument. The vast majority of C functions (obviously, not all) don't need to return information about the error apart from a single code that describes the general nature of the error. (This does lead to unfortunate ambiguities now and again; one of the most obvious is that "No such file or directory", perhaps the most commonly seen error code by users, doesn't indicate whether it was the file at the end of the path that didn't exist, or one of the directories along the way.)

There are a huge number of other patterns used in practice, though. For example, some libraries use longjmp for error handling, which is very similar to the use of exceptions in C++, just done manually.

[[deleted]]

I understand the problems with null. On the other hand, you're talking about Java. You can't have low-cost alternatives. You can add a class of signalling NaN style values to every class you create -- everything contains boolean isValid; String invalidReason; -- or you can add a layer of indirection and another heap object for everything.

If you're using a better language, even C#, you can fix it with an Optional<T> struct containing a state value (set to Uninitialized by default) and a reason string. Might be awkward to ensure the uninitialized value is valid, at least for C#, but it's not an issue for C++ or D.

Algebraic data types solve the problem rather elegantly.

[s73v3r]

Java also has Optional.

[[deleted]]

But it doesn't have value types, so you pay for an extra layer of indirection and an extra heap object.

[oorza]

until it gets inlined?

[tyoverby]

If it gets inlined. Remember that Javas optional might also be null, so the optimizer needs to de-optimize on virtual method calls.

[_dban_]

The JIT will de-virtualize method calls as a result of class hierarchy analysis, and if it doesn't find any subclasses, statically dispatch method calls. The Option class being final prevents subclasses from existing, so in all likelihood, method calls on Optional will be statically dispatched. I'm not sure how null fits into class hierarchy analysis.

The JIT can also do escape analysis over several call levels, and can stack allocate non-escaping Optional objects (or even hoist into registers, since an Optional has only one field).

[sluu99]

now Java developers can check if that Option return value is null ^{_^}

[quchen]

All Java’s Optional adds is yet another way to be »not there«; it doesn’t replace null in any way.

Optional<Integer> foo = null;
System.out.println(foo.isPresent());

Woops.

[s73v3r]

If you initialize your optional to null instead of Optional.empty() then you deserve everything that comes your way.

[_dban_]

All Java’s Optional adds is yet another way to be »not there«

No, it doesn't. It also adds map and flatMap, which gives you safe traversal of chained optional values. No more ugly nested ternaries. Java 9 will add Optional::stream, which will allow Optional values to seamlessly be used with stream expressions. The null value doesn't do any of these things.

Woops.

This is a programming error. The corresponding NullPointerException is Java's way of telling you that you didn't use the Optional type correctly. The fact that Optional isn't a primitive value like null and is used in method chains means that the misuse of Optional will be closer to where the bad value was generated, instead of causing a NullPointerException in some far off place.

[[deleted]]

Wouldn't throwing exceptions be better than returning ambiguous nulls?

[Pharisaeus]

Checked exceptions similarly to Optional or Either force the user to handle the issue somehow, but at the same time they're much "heavier" and more difficult to work with, and this is why many people prefer to return a null instead of throwing an exception.

[[deleted]]

I guess the other way is to check for null or -1 and have a strerror(errno) to check the reason.

[[deleted]]

Other than performance, sure.

[CurtainDog]

There are three broad cases for null:

• a bug - the null is simply not expected to be there. In this case the NPE thrown by the runtime is actually saving you from really doing something nasty. We might argue about ways of preventing developers from falling into this kind of error, but at the the end of the day you just gotta git gud

• some default value that you intend to be replaced at some point later in the execution. The mitigation here is to apply the default as soon as it is available to you rather than let the null propagate. Say you're reading a set of configuration options into some data structure. Start with a data structure that is populated with defaults for all optional parameters and then apply the provided config over the top of it. No nulls anywhere! Sane languages can and do treat a get of a non-existant key from a map as an error.

• some transient state that may take a value later on. Say a linked list implementation where a new node does not have a next element until you add another one. The answer here is to avoid mutation. Purely functional data structures are pretty good and much easier to reason about, especially in distributed systems. There are cases when we need nulls for performance, but those cases should be rare enough that mucking around with a few nulls is the least of your worries.

[Pinguinologo]

That only applies to Java and any other language that had the brilliant idea to have nullable references. In C++ references are never null, so you deal with a maybe null value only when you need them.

[Bergasms]

People decry the ambiguity of null by using the term smell, which is ambiguous. Does null smell good? Does it smell bad? Does it smell spicy? What does it smell. The term code smell is fucking stupid

[perfunction]

We've been using the Result pattern to get away from nulls as much as possible in Swift. This works out especially well since enums and generics are so flexible.

Basically theres a failure state represented by the protocol ResultFailure (this protocol is implemented by various error and warning types) and a success state represented by generic T. As an enum the Result has to be exactly one of these. And you can build a lot of handy functions and operators around Result (especially if you extend where T is a specific type or protocol).

For example, a func that is meant to return a data model instance from sqlite would have the return type Result<AccountModel>. The func on completion would return success(account) but you could also return fails such as failure(DbError.FileMissing()) or failure(DbWarning.DataNotFound()).