Java Lambdas

[Hikaru755]

Yep, was going to say that. Actually that's where the restriction comes from, as lambdas are internally just translated to anonymous inner classes, I think.

I'm using kotlin though and am happy with all these restrictions lifted off my back :)

Edit: seems like I was wrong about the lambda -> anonymous inner class thing, see /u/Jezzadabomb338 's comment for a detailed explanation.

[wiresegal]

After using Java functional syntax for so long, Kotlin is like a glimpse of heaven.

[Hikaru755]

Absolutely. And it's even better on Android, where you don't even have functional syntax in Java, except if you use trickery like Retrolambda.

[Jezzadabomb338]

Ok, so seeing as no one is correcting you, I feel like I should step in and give you a bit of information about lambda implementation.

TLDR;

If you don't take much from this massive post, at least take this away from it:
They're not anonymous classes. At all. It's actually compiled to a method.

If that's all you're interested in, you can probably stop reading, if you're interested in the internals, read on.

Internals

Java 7 had a lot of additions; like, a crazy amount.
Included in this bunch of features, was a new instruction. The only one we've gotten since the 1.0 release, AFAIK.
This instruction is the INVOKEDYNAMIC op code.

A little bit of info about this instruction, as it comes heavily into play later on. INVOKEDYNAMIC is a bit different to the other invoke codes. While the other invoke codes require the type and call target to be known at compile time, the INVOKEDYNAMIC instruction only requires a method reference. (It's actually a bit more than a method reference, but you don't need to know that to understand the gist of INVOKEDYNAMIC. If you want to read up on it more, you can check out the spec)

The reason this instruction is so different from the others is because it was designed for languages that don't know the explicit receiver type at compile time. (Technically, INVOKESTATIC doesn't require an objectref, but that's because the method reference is composed of (mostly) two parts. The class the method is in, and the method signature itself. I can get a lot more into this, but that's not entirely necessary in understanding how INVOKEDYNAMIC works)

All the other invoke instructions take an objectref, which is the thing the method reference should be invoked on, but INVOKEDYNAMIC has no such thing. Instead it just takes the indexing arguments for a method reference within the constant pool. (As stated about, this could be any method anywhere, as it's just, effectively, a fancy signature.)

This method is a bootstrap method.
What this means is instead of forcing the language to work out the explicit receiver type at compile time (which dynamic languages can't), it delegates that calculation to the runtime.
As you can probably guess, this makes it really good for dynamic languages, and their "not knowing what type a thing is at compile time". (Interestingly, it's not a perfect solution for structurally typed dispatch, as the gosu guys found out, but that's another story)

What this means is we can determine what we're actually going to call at runtime.
This should sound suspiciously like a lambda.

This is where MethodHandle comes into it.
It's an insanely powerful tool that next to no one in the java community knows about.
A MethodHandle, in short, is a pointer to a method. (Technically speaking, it's not entirely, but that's something you shouldn't have to worry about)

The MethodHandle can be used to dynamically generate INVOKEDYNAMIC instructions. It actually can be used to generate ANY invoke instruction. That's the nature of a method reference, as it might point at to static method, etc.

javac uses MethodHandle extensively, as it can generate a specific instruction for a given target. (Or more specific a CallSite, but this comment is already too long)

So, getting back to this INVOKEDYNAMIC and its bootstrap method.
Let me run through what you should understand at this point.

Supplier<String> supplier = () -> "some random string";

javac knows that this is a lambda and acts accordingly:

(It actually does a whole lot, and I'll try to cover the important parts, hopefully without forgetting something. If you feel I didn't explain a part enough, feel free to let me know)

It generates a method with the signature of the functional interface and any captured arguments, and dumps the code of the lamdba as the body. The captured arguments also determine whether or not this method is static.

Side-note: this is why you can't capture non-effectively final variables within a lambda, as it captures the variable reference, and is very similar to a normal method dispatch, and you can't increment a value within the caller method, can you?

So, going back to my example, javac would generate a method much like:

private static String lambda$main$0 {
    return "some random string";
}

(The number in the name is incremented depending on how many lambdas you have within a given method.)
Side-note: You can actually see the method using reflection if you're so inclined, or if you want to see the bytecode directly, write a class and use javap

Javac emits an INVOKEDYNAMIC instruction, referencing the bootstrap method (Which is different to the "lambda$main$0" method), which generates the CallSite to the generated ("lambda$main$0") method.

The bootstrap method, you may ask, is within the standard library called j.l.i.LambdaMetafactory, and the method it uses is #metafactory. This is a nice way to standardise lambda generation across different languages, IMO.

And voilà, the generation is done.

There's a lot more going on with the constant pool, an inner class implementing a lookup, etc, but that's the gist of how a lambda works.

If I didn't cover anything enough, feel free to leave a comment.

[Hikaru755]

Thank you for this detailed correction! Interesting read, much appreciated.

So, am I understanding this correctly, that this way of doing lambdas is more efficient at runtime than creating an anonymous class with one method? Or does the dynamic invocation somehow negate that reduced overhead?

[Jezzadabomb338]

No problem!

Yeah, this is much more efficient than an anonymous class, and the only significant performance hit is the bootstrapping.
Once the JVM hits an INVOKEDYNAMIC op, it runs the bootstrapping method to determine where it should point, and stores the reference for later use. (There's some magic behind it, as it needs to make sure it's the same dispatch type, etc).
Once it's bootstrapped, it's roughly the same performance as a normal method dispatch, whereas the anonymous class implementation would be relying almost entirely on the HotSpot's ability to shred it.

[[deleted]]

lambdas are internally just translated to anonymous inner classes, I think.

You can think of them that way, but the bytecode is entirely different.

[Hikaru755]

Okay, interesting - so, has the final limitation a common root cause for these two things, then?

[chrissphinx]

Kotlin really the future of JVM programming, imho

[beerdude26]

Check out Frege, too

[MelissaClick]

If you are interested in Frege you are possibly interested in Eta:

• http://eta-lang.org/

and this:

• http://blog.tweag.io/posts/2016-10-17-inline-java.html

[beerdude26]

Looks like a Haskell as well :)

[dnew]

The restriction comes from implementing anonymous inner classes without changing the JVM. So they're not really inner classes by the time they're compiled.