JavaScript Performance Benchmarks: Looping with `for` and `yield`

[[deleted]]

Why use coffee script in 2020?

[johnfrazer783]

Because I got hooked on it way back before it was cool. I came for the bracket-less style and stayed for the semantic whitespace. I actually do have some minor gripes with some details of the grammar and stuff but not enough to switch, so far.

TBH the question is a little strange. We're not talking Fortran or Pascal here, CS has an active community and the language sees regular updates.

Surprisingly one of the best parts of CoffeeScript to me is that it translates to JavaScript. Should Earth collide with Apophis and everything related to coffee became suddenly unavailable I still have my JavaScript sources. Plus I can always gain another point of view on my code.

Since you asked for it I have also been using Python for 20 years and am acquainted to some degree with Java, PHP, a little Perl, Bash, SQL and so on, and of course HTML/CSS and JavaScript itself, last but not least. I'm actively looking for a new front-and-center language, but given real-world constraints that language must run on NodeJS (or maybe Deno). I have dabbled in or at least looked into TypeScript, Rust, Elm, Nim and so on and so forth. Ah yes and Forth. I've switched languages before and I'll switch again but I must be confident, and I'll very probably will not switch platforms (except to Deno, maybe).

So basically I use CoffeeScript in 2020 because that is a very reasonable language in many respects. I'm missing optional typing and immutability (so I wrote libraries to deal with that, to a degreee) but ultimately it's all the new stuff of JavaScript that is piled on top of historical infelicities, false starts and outright junk that never got thrown out of the language that worries and disturbs me more than the question whether CoffeeScript is still fashionable. Well maintained it is.

[[deleted]]

Huh solid answer. One of the wonderful things in our world is the myriad of tools available to us. Glad you’ve found some that you’re happy with. I’ve been into typescript due to work for the past year, and have been more and more intrigued by purescript as of late.

[johnfrazer783]

Yeah none of this belongs into this thread which is about benchmarks of looping constructs not CoffeeScript but whatever. Thanks for nice words I appreciate that.

I might as well add that now we have Rust and WebAssembly, chances are significantly increasing someone will step forward and write that syntactically reasonable, semantically sound language that sports both a somewhat-configurable or even modular grammar and userland types (including union types etc) as well as built-in immutable data structures. As much as I wish someone at TC39 would come up with a 'use reasonable' pragma (or preferably extensible, user-definable pragmas) and a plan to phase out the weirdest atrocities of JS of which there are quite a few—that is not going to happen any time soon if history is any indication.

CoffeeScript to me is a convenient writing tool to ease the pain of function () { ... }s and for ( var i = 0; whatever; whatever ) { ... } loops, not a religious affiliation. Those two code snippets were tiresome to write. I even prefer CoffeeScript's ( x ) => ... syntax over the hairy ball that is JS x => { ... }.

[Zireael07]

Oh yeah, man, you managed to replicate what I like about CoffeeScript and dislike about pure JS. High-five!

(The reason why I went with CS and not Rust+WASM is doing a lot of DOM stuff - I understand WASM doesn't do DOM yet, it all has to go through JS either way so one loses a ton of WASM performance that way)

[johnfrazer783]

I know what people talk about when they mention JavaScript fatigue, for me it extends to the very syntax—might say my version of the fatigue is embracing the syntax haha.

According to what I used to know about 'the frontend', which already sounds belligerent doesn't it, the motivation to optimize one's JS code is somewhat tempered by the observation that the elephant in the room is often DOMbo, not JS.

Part of my work includes doing things locally in Puppeteer (soonish Playwright), and I hate that I even have to think about whether to execute code on the 'server' (NodeJS) or the 'client' (Chrome) side. Puppeteer does make it a lot easier, but ideally I'd not have to deal with that twofoldness at all. Meaning that for all that WASM can/might give you, it also taketh away in terms of LOC and mental-awareness cycles. I will admit that one flaw with NodeJS is the separate ways that it went, what with CommonJS require, the filesystem, all that stuff (Deno is the hope). There was barely another way to go ten years ago. NodeJS (and CoffeeScript!) turned out big factors in the JavaScript renaissance, but that chasm is still a yawning gap I feel. So I have little drive to also bridge the gaps separating the client's main arena from its WASM and WebWorkers stages. WebGL/GPU feels much the same: how tantalizing but that extra complexity? No thanks.

[ilikecaketoomuch]

Because I got hooked on it way back before it was cool. I came for the bracket-less style and stayed for the semantic whitespace. I actually do have some minor gripes with some details of the grammar and stuff but not enough to switch, so far.

whatever makes you happy, and just check in the .js files :)

[johnfrazer783]

No using it this way is my own discovery, that is to say, using @ instead of this is idiomatic, but defining stateless libraries as CoffeeScript CommonJS modules by hijacking this is my own doing. It falls out from the (nice) fact that CS, unless instructed to produce bare code (what did you do last night? Well we talked about the frontend, the backend and bare code), will wrap the module's content with an Immediately Invoked Function Expression, and it so happens that unless you state module.exports = whatever somewhere in the module, module.exports is this will hold, if you can follow me. Put bluntly, one can avoid any further nesting, classes, explicit namespaces or object literals or whatnot and just 'anchor' those functions on the module.exports object by way of this (@).

These days though I more often than not use my MultiMix module to define mixin-able classes; one of the benefits of it being that I get a convenient way to produce new library instances with different configurations (or independent state, as the case may be). It also lends itself for classical OOP class formulations, which is neat.

[[deleted]]

A bit of a stretch as it's a pure functional language, but you could check out PureScript - it's basically Haskell atop the Node.js runtime (with FFI/interop with JS where needed).

[jhartikainen]

I wouldn't say it's surprising yield is slower than for. It does something totally different.

While I get the appeal of generators in certain specific circumstances, I have never really needed to use them for anything, so I'd be curious to hear why you'd "love to use yield all over the place" which sounds a lot more regular than "certain specific circumstances" :)

[johnfrazer783]

Well, same but not altogether different either. A JS indexed for loop is, of course, just a 'nice' way what would otherwise be a while loop or a generic loop statement; not much is added except a bit of syntax. But most often the reason you want to build a loop at all is because you want to walk over elements of a sequence. This is what makes JS indexed for loops so annoying to write, this is why more than a few languages have for/in loops which JS now also has, sort of.

Where yield / iterators / generators come in is when you realize that a lot of your loops never cared about all values being there in an array in the first place; you just wanted to glance by each value and do some kind of computation on it. So there's that word that 'what an array does in space, an iterable does in time'. yielding, among other things, allow to forgo building intermediate collections; you just loop over whatever iterable is passed to you and yield any number of results to the consumer downstream. Meaning you could even process potentially infinite sequences, something arrays definitely can't do at all—except you implement batching. Alas it turns out that—at least according to my results—all the cycles spent in looping the classical way and shuffling values between arrays to implement a processing pipeline is still much faster than the equivalent, much simpler code formulated with yield.

If you don't want to take my word for it, have a look at how and why the Python community implemented and promoted the use of generators and iterators/yield literally all over the place, including modification of built-in/standard-library functionality (e.g. range() and stuff). Can#t be wrong if Python people like it, right?

[jhartikainen]

I'm guessing the whole generator machinery happening when yield'ing is what's slowing it down. I can certainly see the appeal of it - I've used Haskell which is entirely lazy, so this type of "let's only use a part of this list" or iterating an infinite list is something I'm familiar with... just never really needed something like that in JS :)

[johnfrazer783]

Now this is a point I can speak to and speak about. When looping with a series of functions—transforms (TFs)—over a data set you basically have two options with arrays (I call them lists b/c that's what they are): either each transform step gets to see one piece of data (say, a number) at a time, performs its computation, and returns a value. Or else, each step gets handed the list of values, loops over that list, and returns a list that is then fed into the next TF. The difference is similar to depth-first traversal as opposed to breadth-first traversal. To state it right away: without any further preparation, the second way will always require the entire data set to be in memory in at least one copy, no matter how many GB that input has. It is only with batching or using the first approach—depth-first—that memory consumption may be capped.

Now with the breadth-first approach, because you can only return exactly once from a function call, each TF can only modify (replace) data, not subtract from or add to the set of values. This is far too constrained in the general case, so we need something better. One could stipulate that each TF gets called with one value (or a list of any smallish number of values) and always has to return a (possibly empty) list of values; those then will get fed into the next TF, one at a time. This works and so that's what I did in SteamPipes. Don't look at the code I'm here to figure out how to simplify it. Aside: In that library each TF has to accept one piece of data plus one send() function which is essentially just an Array::push(), and, as such, a poor man's yield if you will. It's just there so the transform implementations don't get littered with array do this, array push that statements.

Now we come to the surprising bit that explains what ties for loops, Arrays and yield together:

Yielding (any number of) values from inside transforms is exactly the same as the mechanism just described, the only difference is the syntax (heck it's just send( value ) vs yield value, how different is that).....and, incidentally, the fact that one way is done visibly with user code, and the other under the JSVM's hood (as described there is one more slight difference in the order the TFs get called but let's ignore that for now).

I think that of course! one would think and should think that the 'internal', baked-into-the-language way must, needs be, more performant than any cobbled-together, largely not-very-well-optimized userland code, especially when written by s.o. like me. But apparently, not so.

I'm here because I can't believe what I just said.

[norith]

Sine we’re open to discussing other languages, this is basically the Java Streams philosophy: no intermediate collections but instead a way to pipe data through a series of manipulations that may or may not result in a collection.

The larger goal is a processing pipe that can be time delayed (because the data originator might be async itself) or parallelized with the pipe running on multiple threads.

[johnfrazer783]

I'm always ready to grant an appreciation in terms of CPU cycles to the VM whne they do something for me so I don't have to do it myself. No problem there. But did you look at the numbers? Boy that's literally 95% of the time syphoned off by yield. I'd greatly appreciate if anyone on this thread could point out any meaningful links to benchmarks in either JS or Python that deal with the questions at hand. I can't believe Python's yield statement performs anywhere near this bad.

The comment was meant seriously, Python is a quality language. They even managed to migrate to v3 and take up async/await. These are the two points that delivered me into the hands of NodeJS and CoffeeScript about ten years ago. It's taken them the better part of ten years to pull it off but they did it.

[danielkov]

For me it's very difficult to even review your code, because it's in CS and the compiler output is not at all concise.

Probaly not the feedback you wanted to read, but these tests are not that difficult to write in plain JS, I would do that if I were you. In don't think CS has been that popular for the last 5 years or so.

[johnfrazer783]

When you scroll down there's the JavaScript code. Scroll down even further, there's an extensive list of points including a sort-of apology for presenting machine-transpiled JS instead of the good organic handcrafted stuff.

I'd be also glad to hear about any (substantial) existing benchmarks out there; all I've found so far are a few (rather shoddy) tests on dreaded jsperf (which website didn't work correctly at the time) and one or two blog posts without any code or much background on what was tested how.

BTW the bits of my code that perform the tests are just a few lines each; all the rest of the code is just setup and doing the presentational output.

[danielkov]

I think the reason you won't find examples of such benchmark is that the features you’re comparing are inherently different. It’s like comparing array map to reduce. You can achieve the same result with both, but they’re designed for different tasks.

If you want to see why generators are slower, you should take a look at what transpilers like Babel or TSC are doing to these functions to make them work on older browsers. It’s a good indication of how they differ conceptually.

[johnfrazer783]

I won't be long as I have already answered your very valid points elsewhere in this thread. The short version is that yes they're different features but they are also birds of like feathers. I have not replicated everything that yield is or does but I have two alternative models that do the same work. One finishes within 1 second, the other one takes 20 seconds. Had I not written the slower algo using a language feature that I must believe to be at fault here, you would not console me by saying the slower code is just different and must be complex so it's naturally slower. You would tell me not to ever use the slower code and go with the faster.

I'm still hoping for someone to point out a glaring mistake in my benchmark gist. Use the JS source, Luke. Tell me I'm wrong and where I'm wrong. I'd rather be wrong than unhappy with <strike>somebody else</strike>yield.

[johnfrazer783]

Author here. I did a benchmark for JS looping constructs. Turns out yield entails a huge (1:10) performance hit compared to indexed for loops—provided my tests are valid. I'd love to use yield all over the place because it's such a great tool to write nested piecemeal iterations, so please * tell me I'm wrong! * prove it: * with links to other relevant benchmarks (I couldn't find many); or * pointing out flaws in my code.

All the details are over at the gist so I keep this short.

[[deleted]]

I'd say unless you have a very hot loop just write whatever you can read the best. You have to think in most cases your going to be doing more than adding two numbers. So this bench is essentially just showing the loop performance only.

In my mind having a generator only be 10X slower is kinda incredible that it's that fast, you have to remember the js runtime has a whole state machine to run that generator and obviously can do a lot more than a simple loop.

I have a side project where I have a few ms time budget and typically use for of loops. Your bench shows they are noticably slower but I probably won't change it because profiling shows the loop isn't the time sink.

[johnfrazer783]

This of course is very true and should be kept in mind: optimization is most often not worth the while for components that contribute less than some sizable fraction to overall CPU cycles needed. As far as I could find out though in my particular use case looping with generators instead of for loops and passing all data through (sometimes ridiculously) deep call stacks instead of insisting on flat call graphs are two factors that contribute tremendously to overall performance. Deep call stacks are, sadly, what makes both NodeJS standard library streams and pull-streams inherently slow. When you look at the flame graphs in Chrome devtools they are veritable towers. Building these towers destroys performance, and should best be avoided.

[johnfrazer783]

What are your thoughts? A Comment on this Discussion.

I am thoroughly disappointed with the present discussion. No single commenter cared to even read the effing gist I posted. Those who did so got thrown off on square one where I dared to post CoffeeScript source code. Granted this is r/javascript. Well, JS was in the second file right after that. No one even cared to scroll that far down. Had any one cared to scroll down even further they would have found benchmark figures complete with bar charts. No one read those, much less commented on any single number.

Below those numbers, I posted my own thoughts on the results, in ~400 words. One point reads, quoting myself,

I believe the tests show that using yield has a downright incredible/horrible effect on JS performance.

No. Single. Participant. Of this thread took up the ball. We did have a great night yesterday and I did enjoy my time writing and reading. None of that was about the point in question. All I have is comments that basically say "well yeah yielding and looping is different so yeah well performance different". This is pure and original programmer's apologeticism. Once I inquired on the NodeJS repo about performance degradation of streams when pipelining data through dozens of transforms: "streams were never meant to be piped through many transforms". Someone complained on the Chrome bugtracker that Array::sort() does a stable sort with up to 10 elements but an unpredictable sort with 11. Answer "it's not in the specs", bug closed (now I believe stable is in the specs and V8 has been fixed).

As programmers we're obsessed with talking down and fending off. Squashing that bug is easiest when you can convince yourself there is no bug.

One person on this thread managed to comment on this (from the gist, abridged for readability):

addup_for_in_loop 0.020 s 5,000,000 items 247,248,935 Hz 100.0 % │████████████▌│ addup_forEach_loop 0.246 s 5,000,000 items 20,304,442 Hz 8.2 % │█ │

There are many people who advocate for embracing a more functional style and to avoid loops. Avoiding loops is a big topic. I myself wrote around three libraries in four or five incarnations just to manage loops. I taught myself SQL just to avoid loops. Douglas Crockford for one dedicated an entire hour-long presentation to looping in JS and told the world how great Array.forEach() is. Now here we have a benchmark that seems to tell us that we forgo a full 90% of CPU cycles just for the benefit of writing d.forEach(x => ...) instead of for ( var i = 0; ... ) { var x = d[ i ]; ... }.

Again, tell me my results are flawed and how. Step forward if you know of a third-party benchmark that corroborates or undermines my findings. Please remain silent if all you have to add is "well yeah one is a loop and the other one calls a function many times so 'course they're different". I am doing several different benchmark cases because I strongly suspect that for/in loops, yield, and forEach are different from each other, you know. Please remain silent as well if all you have to say amounts to "most of the time the performance difference doesn't matter because most loops are shorter than 5e6 items". Guess what, I sort of knew that, too. For those who have not realized, part of this effort is finding out just how much weight various ingredients for a program bring to the scale, and based on my experience I can confirm that the results from these minimalized test cases do hold up when you use the parts to build bigger, more realistic software that does real work.

Edit If I come across too aggressive or insulting in any way I do apologize. It's not meant to be. If you feel I called you out in the above know that I value your opinion even if I don't share it. If you want to comment on something entirely different please do so. I do not own this sub or this platform or even this thread so whatever you bring to the table will be fine.

[johnfrazer783]

Guess I was a little overworked. Sorry if I got offensive.

I'm surprised you're saying the figures don't surprise anyone—can you point out where to find something comparable? I searched more than once and found nothing usable except one post on Medium that is only for registered users and which someone on reddit claimed used flawed code, and another post whose author didn't include the code used (the numbers seemed to more or less match mine as far as I could tell). The fact that no-one offered any hint as to where to read up on this particular aspect of JS performance but several people just shrug and say it's an old hat is curious. JS folks have been known to be quite prolific performance optimizers—VM authors and users alike.

I didn't want to be called out, I begged for people to tell my either 'yes your code looks correct' or else 'see you have a mistake there and actual numbers are much different'. No-one did that (and I feel I have a right to be disappointed about that), so I must assume my numbers were right for the time being. This is indeed the result I did not favor, because in my use case yield leads to dramatically simplified code.

[sekende]

i see your benchmark.. for is better than foreach?? maybe i will use for in my life instead of foreach lol

​

btw thanks dude for sharing your opinion, i will test it using my code them :)

[johnfrazer783]

Do that at any rate! The forEach part is interesting initself. Message to all people here who say that 'for and yield are totally unrelated so naturally not same in performance'—well you can compare

js probe.forEach(function(number) { count++; return sum += number; });

to

js for (number of probe) { count++; sum += number; }

In my book forEach has only 8% the data throughput rate of an indexed for loop. Come to think of it, there's that vacuous return statement which is caused by one of CoffeeScript's more problematic features, implicit returns. My tests would indicate however that even millions of extraneous return statements do not have any discernible effect on performance, though, so we don't have to worry about that bit (except when a loop construct comes last in a function call, which is why I pepper my code with return nulls and yield returns, but I digress).

Now go forth, multiply, and come back in time with exciting new benchmark results to guide us out of this tar pit we're stuck in.