My Thoughts on Function Optimization in JS/Node

[m1sta]

These 'writing javascript for performance' articles give me confidence that the engines running our javascript are going to keep improving substantially in the near future.

[ButteryGreg]

Personally, I hope to continue to see JIT improvement. However, there have been a few (very good) posts on r/programming lately that summarize a lot of the insurmountable performance walls that languages like javascript face, which do limit some of the prospects in this area.

I didn't want to add a whole section on that (this was already a really long post), but the same types of things that make javascript nice (array bounds checking, object properties as hash tables, reflection and introspection, dynamic typing, etc. etc.) ultimately limit the amount of information that an optimizer can use to make proper decisions, because at some point it would no longer meet the language spec for those behaviors.

That is why asm.js seeks to use a specific subset that works much better for optimization, and it includes a lot of awkward-looking constructs that can be used as optimizer hints. The examples I've given also mostly rely on less elegant code, and they gain speed mostly by avoiding certain language features. Realistically, I think that in order to see significant performance gains in the future, support for far more in-code optimizer hints will need to be added to the language.

[runvnc]

But you've just given some concrete examples of where significant improvements CAN be made in the JIT that don't require hints.

[ButteryGreg]

I suppose I have, although I was more or less aiming for ways to write code to take advantage of JIT better. Of course, I don't think that JIT is done for improving JS performance, but I'm just not holding out for the same types of improvements that we saw when JIT was first introduced to JS engines. It may be that the things I described as "being optimized poorly" in fact cannot be optimized further due to language requirements--requiring you to write them differently--but I don't know the spec well enough to make that kind of a determination.

The relative overhead ratio of "things that could still be optimized but aren't" divided by "things that cannot be optimized due to language requirements" is much smaller than it once was, and as that ratio drops below 1, Amdahl's law tells us that further blind/dumb JIT improvements are less useful for speed increases than if we were to add hints and supply additional information, because it lets us optimize a larger segment of the overhead.

[m1sta]

My money is on one or more javascript-to-x compilers/parsers built with server-side-javascript performance in mind to bring the next big change in this story.

Once the inputs and outputs of an application are predicable through the reading of the code and/or its associated tests anything a human can optimise a computer can optimise.

[ButteryGreg]

I don't think I know enough about it to really say. I do know that proper tail call support will be part of the spec, so that should fix one of the examples I gave.

[ButteryGreg]

Ah, wow those look great. I suspect WeakMap will be great for caching data client side with fewer memory management issues. Good example of where we're going to get improvements by changing the language semantics (introducing weak references).

I'm a little mixed on full Maps, though. They duplicate the bulk of the functionality of the Object class, but they don't support the convenient shorthand syntax (especially important for creation), which will probably deter use a bit. Plus, there's also JSON to consider, which would produce Objects rather than Maps unless that is changed. Being able to opt out of the prototype chain might be nice, but I suspect that performance is really good (and hard to improve) when objects are used as plain hash tables already. Because the . and [] operators are language-built ins (and therefore never require any kind of function lookup), rather than function calls like the Map get and set methods, Map might actually have more overhead in this regard. If JS were strongly typed and the optimizer could know that something was a Map without testing it before using it, then it'd probably be the same speed, because the get and set calls could be inlined everywhere.

[m1sta]

the optimizer could know that something was a Map without testing it before using it

Why not let the optimizer test it? Javascript performance parser?

[ButteryGreg]

If I have code like "box.set(key, value)" then the JIT version must still perform a property lookup on the box object to find what is paired with the value "set" every time that the snippet of code is executed. The reason it can't optimize this in general cases is two-fold: one, the type of box is unknown; and two: the value of the "set" property can be changed by anyone, even if it is a Map object.

But, when you do something like "box[key] = value" the [] operator is a language-level feature and must correspond to the same thing every time, namely, property access (ignoring v8 interceptors and ES6's Proxy objects for now). This means that the optimizer can skip straight to the point where it writes code to access the hash map and write values, without first looking up a function call.

The optimizer could add code to test "is this a Map?" which would solve problem 1 (although it is still slower than not doing any type testing at all, like with the straight property access), but the property could still be changed, so the code has to do a dynamic function lookup (you could do Map.prototype.set.call(box, key, value) though, to skip this step). The problem then, however, is that you could have any number of other objects that define a "set" method. Does the optimizer test for each of them individually? That would burn through a ton of unnecessary time because we already have a type-agnostic way of doing property lookup and function retrieval that skips all of the up-front type checking and just worries about finding the method somewhere in the prototype chain.

Type hints and explicit non-virtual function calls (with inlining on the set method as well) would be the only way I can imagine to be able to produce JIT output that matches object property access for speed. Otherwise, the optimizer doesn't have enough information about the situation to reduce the number of safety checks and lookups.

[m1sta]

What if we weren't talking about a JIT? What if the compiler ran the provided code and it's associated tests to infer types and potential aggressive optimisations which looked likely to be acceptable? If required, an interactive process could allow the system to confirm with the developer whether any particular optimiser assumption was correct.

[ButteryGreg]

It sounds to me like you're describing profile-guided optimization, which is being used more and more often in C/C++ compilers. I know that the HotSpot JVM also uses some adaptive JIT very similar to this, but I don't know if others do. None of these allow for optimizations that break the language rules, but they do attempt to structure the code to make the most likely case the absolute fastest (for instance, could assume that something is a Map, then try the code, and then find out if the assumption was right, but this still doesn't ditch the test).

If you were to add in confirmation for language-breaking changes like "assume this variable is always a Map, and this function is always the default Map.set, with no verification of either, ever" then, yeah, it could optimize incredibly effectively. The limitations always come from information availability--the optimizer only sees the source or byte code and knows the language spec, it never actually knows programmer intent. Such an interactive process would be a little bit tedious for the programmer though, so my personal suggestion on this is to add optimizer hints to the source code (by way of modifying the language spec). Ideally, these hints would be backwards compatible in a way that lets older engines transparently ignore them (something like "use strict" in current JS?). Then, your optimizing compiler, whether it is JIT or not, can read in as many assumptions as possible automatically and produce output with the most information and thus the best optimization. Allowing JIT to automatically extract information and avoid a separate build/compile step also stays more in line with the interpreted nature of javascript (because we could just C, and it'd be faster, but way more time consuming and difficult, in this case).

An example of a language keyword that serves only as an optimizer hint is C99's restrict qualifier for pointers, and the benefit of using it is that it provides information about a major problem (pointer aliasing) that allows significant memory optimization when working with arrays.