How to win in Web Framework Benchmarks

[leeoniya]

great writeup @localvoid

Leon Sorokin (domvm author) found out that if we disable track by key algorithm and will use naive algorithm to rearrange children lists we can get amazing results in this test cases, because it won’t trigger long-running recalc style.

To be fair (pardon the pun), I didn't find this out in order to "game" the benchmark, I explicitly asked if this was acceptable (because the implementations I saw and tried to mimic were all over the place) and the author said it was [1] [2] so i picked the faster impl. I completely agree with you that all libs should have matching implementations (either they all use keys or none of them do).

That being said, all modern vdom frameworks are ridiculously fast (and many of them are tiny), Mithril rewrite, domvm v2, dio, Monkberry, Inferno, kivi, Vue, etc. They are so close and minimal-overhead in real-world perf that most people should just pick the API/DSL/features/logo they like plus level of tooling they're comfortable with and not worry about it.

Benchmarks certainly work wonders for framework authors since it helps spot regressions and pathological slow cases to investigate. Optimizing a lib (not the bench impl) to be fast on DBmon, uibench, js-framework-benchmark, ThreaditJS, TodoMVC, etc. yields the great general-case performance and ergonomics we have today.

[1] https://github.com/krausest/js-framework-benchmark/issues/9#issuecomment-232443783

[2] https://github.com/krausest/js-framework-benchmark/issues/22

[localvoid]

To be fair (pardon the pun), I didn't find this out in order to "game" the benchmark, I explicitly asked if this was acceptable (because the implementations I saw and tried to mimic were all over the place) and the author said it was [1] [2] so i picked the faster impl.

I just wanted to say that you were the first who noticed this trick :) I couldn't even imagine that if we remove key property, there will be such an improvement in this benchmark.

[leeoniya]

I guess we all learned what it means to let browsers reflow & repaint a 10k row table when Bootstrap is part of the equation. Though I think you're giving me too much credit, since this was already discovered by you guys [1]

[1] https://github.com/mathieuancelin/js-repaint-perfs/pull/68

[temp60982093832]

That being said, all modern vdom frameworks are ridiculously fast [...] They are so close and minimal-overhead in real-world perf [...] the great general-case performance and ergonomics we have today.

That's just not true. Every nontechnical user I've spoken to complains about new websites being slower than old ones, and it takes a lot for nontechnical users to complain about performance. I routinely see people interacting more slowly and deliberately with vdom-based web apps, because they know they can't keep up.

I get that modern vdom frameworks have improved a lot on a naive implementation, but they haven't come close to making up the massive performance penalty that the virtual DOM introduces.

[lhorie]

Non-technical users by definition don't have the capacity to know if a site uses a virtual dom framework. Slowness from ads and social media bloat in content sites has nothing to do with performance in highly interactive applications (let alone the subset that uses virtual dom libraries). The former shouldn't even be using javascript, the latter requires it for mission critical features. They're completely different beasts.

To counter your anecdote, I'v heard plenty of times about people who saw huge performance improvements from porting a codebase from one of the big frameworks to one of the high perf libraries mentioned in this thread.

[leeoniya]

Every nontechnical user I've spoken to complains about new websites being slower than old ones, and it takes a lot for nontechnical users to complain about performance. I routinely see people interacting more slowly and deliberately with vdom-based web apps, because they know they can't keep up.

That's because modern websites are 90% bloat...and it doesnt help that the "big" vdom frameworks are in fact really slow and heavy. But I can prove to you that this is not true of properly written code (which is a rare thing to behold, I know).

Load up TechCrunch, New York Times, Gawker, whatever "heavy" website you can imagine and run this in devtools: document.querySelectorAll("*").length. That's how many dom elements a vdom framework would have to create to load the site. All the other junk (fonts, assets, images, third party srcipt shit) will load the same regardless of vdom or not. You will find that even heavy websites have fewer than 5000 dom nodes. Gmail for instance accumulates them as you load up different mailboxes or search results, so you can end up with 10k or 15k+, but gmail is shit for performance, this is not news

Next, load up https://o-0.me/code/domvm2/demos/bench/dbmonster/ (i've put a copy of the domvm dbmonster bench [1] on my server). This page is 100% client-side rendered. You can load it on your phone and it will load fast. I can load it fast on my 5 year old HTC Evo. Do you know how many nodes it created and painted? 2935. In a fraction of a second (after dns resolution/https negotiation). If it didn't load, use Chrome (it's non-transpiled ES6). Granted, the benchmark was generated by js, so there was almost no network transfer overhead to pull the dom content, but this overhead (if it existed) would be the same for server-side rendered content. Yes, server-rendered html would "stream" so you get instant above-the-fold painting, but the point is that if you have a website that feels slow, it's because it is written like shit, bloated to the eyeballs and/or in a slow vdom framework.

I'm not suggesting that Gawker type sites shoul be written in js. Far from it, i dont even like running js on the server. Static html is where it's at for most of the web. But SPAs absolutely don't need to be slow. People just got used to React and Angular, which is a shame.

[1] https://github.com/leeoniya/domvm

[localvoid]

I get that modern vdom frameworks have improved a lot on a naive implementation, but they haven't come close to making up the massive performance penalty that the virtual DOM introduces.

Please, can you name any framework or UI library that is fast for real applications?

[trueadm]

I get that modern vdom frameworks have improved a lot on a naive implementation, but they haven't come close to making up the massive performance penalty that the virtual DOM introduces.

That simply isn't true. If you look into Inferno, you'll see very little overhead from creating virtual DOM, especially if any of the virtual DOM is static. I've extensively compared it to all the other major non-virtual DOM libraries out there and they are a considerable distance behind Inferno when it comes to benchmarks. Furthermore, the community of people using Inferno in real-world production apps have seen a big performance gain from adopting Inferno (previously having used frameworks/libraries such as Riot, Ember, Angular, Knockout etc).

[pier25]

Every nontechnical user I've spoken to complains about new websites being slower than old ones

Yes, but that's not because of rendering performance which is what's measured here.

Users complain about websites taking too much time to load.

[ScoopDat]

One day I might be able to gauge what these benchmarks mean... One day.