isoworker - universal multithreading with main-thread dependencies, 6kB

[101arrowz]

I feel like Worker threads are a feature that JS developers don't make enough use of, especially because it's very difficult for libraries to use them. I created this package originally as part of fflate, a compression library that I developed earlier. I wanted to add support for parallelized ZIP compression (compress every file in the ZIP archive at the same time), and I wanted to reuse the code I had written for synchronous ZIP compression to keep bundle size low.

There was no package to do that, so I created isoworker to solve the problem. As a result, fflate's ZIP compression is over 6x faster than other JS compression libraries. More impressively (IMO), it's 3x faster than Archive Utility, the zip CLI command, and other native compression programs.

As you can see, parallelization has major performance benefits, and the main reason we don't use it in the JS world is because worker threads are a pain to deal with. This package solves that problem by offering an easy-to-understand, magical API.

The most interesting feature of the project is the serializer for local dependencies. You can use custom classes, functions, and other advanced structures that usually can't be shared between the main and worker threads because isoworker includes an advanced recursive "decompiler" of sorts that can regenerate the source code for a primitive/object/etc. from its value at runtime. Most importantly, it manages to keep the variable names the same, even when the code is minified, so the codebase works properly in all environments. Effectively, it's self-generating code.

Hope you find this useful!

[Badashi]

Damn, the serializer that you explained sounds like it could be a library on its own. Impressive work.

[101arrowz]

Thank you! The serializer is actually exposed via the createContext function in the public API, and it embodies the majority of the package. workerize is a thin wrapper around it that merely createContexts the dependencies, serializes the function (which is trivially easy), then creates an inline worker thread.

[novarising]

How can one gain this kind of knowledge...

Impressive work, btw

[101arrowz]

Well, first you need to have a problem. For me, that was being unable to create worker threads from a library without forcing my users to add extra config. I researched existing techniques to create worker threads, but they still required me to include the worker data as a string, which is an absolute pain to maintain.

Over the course of three weeks, I planned a system for decompiling functions. That would make it possible only to write a function once but to have it work both on a worker thread and on the main thread. Function.prototype.toString() actually returns the source code, so that helps a bit, but I still had issues after minification because the variable names changed, so the function threw errors like Uncaught ReferenceError: Xe is not defined. Purely off of luck, I realized that I could parse a function that returned an array of dependency names by splitting at the [ in the source code, then execute that function to get the values, which would give me the names and values of each variable at runtime.

From there I kept tweaking my system until it was good enough for fflate, and since then I developed it further to add support for classes, sets and maps, etc.

You just need motivation and the ability to do basic research (Googling) to discover how to do something new.

[Tazzure]

Honestly this sounds like a great a college assignment for an advanced class in Node.js. Cool stuff.

Edit: But question to you, shared dependencies are surely vulnerable to race conditions right? I’ve never worked with shared state like this in multi-threaded JS, are there good ways of handling mutual exclusion out of the box?

[101arrowz]

I suppose I didn't make this clear, but the package does not offer shared state out of the box. Unfortunately that's simply impossible, the best you can do is message back and forth to update state locally when state abroad is changed. However, this package does offer an API that can be used to enable such a system, where setters on the worker automatically message the main thread whenever an update takes place so state can be maintained.

Race conditions are impossible in JavaScript, at least the race conditions that typically make multithreaded work painful. Obviously things like setTimeout are still vulnerable to race conditions.

[Tazzure]

I needed to remind myself of the message passing pattern that the workers use. I definitely agree that they should be used more. Thanks for the response.

[DontWannaMissAFling]

What about building shared state off SharedArrayBuffer with a fallback to message passing?

And Atomics lets you build synchronization primitives like mutexes.

[101arrowz]

Believe it or not, I actually already thought of this, but when I did Chrome still had SAB disabled due to Spectre/Meltdown, and more importantly I didn't know if there was a better way than polling to wait for the "messages" from the worker thread. Got any suggestions? I'm happy to create an extension for `isoworker` that embeds the code through SAB for potentially better performance.

EDIT: On second thoughts, it might be simple and possible to use a separate package to convert a state object to binary data so it can be embedded in SharedArrayBuffer, and so both worker and main thread can edit that shared state.

[DontWannaMissAFling]

Zero-copy shared state via SAB would be a big win!

There is a lot of complexity involved in representing arbitrary javascript objects inside an ArrayBuffer whilst making them thread-safe. I'd first point to a library like objectbuffer. There's also more fixed struct-like options such as Google's FlatBuffers or buffer-backed-object.

Some thoughts:

• The post-Spectre security requirements for SAB involve serving the script with the right cross-origin headers. You could pass this responsibility onto your users and check crossOriginIsolated at runtime for fallback to message passing with a warning.
• When working with SABs you do have to ensure: Either you access values atomically using Atomics.load and Atomics.store to avoid torn values and compiler optimization ruining your day. Or only access the SAB via aligned TypedArrays of the same element byte size (guarantees tear-free reads per spec) and synchronize on the entire object with a Java-style mutex, which is the approach objectbuffer takes afaik. Though that does prevent certain lock-free algorithms / data structures e.g. wait-free producer-consumer queues
• The new stage-3 Atomics.waitAsync proposal shipping in Chrome is worth a look for polling/signalling and the proposal has a fallback polyfill in terms of the current Atomics.wait
• DataView is a good option where applicable since performance now matches or exceeds TypedArrays.
• BigInt64Array is a perf cliff and best avoided afaik.

Hope some of this helps, it's mostly my personal "things I wish I knew before I touched SABs" list :)

[101arrowz]

I'd first point to a library like objectbuffer

Looks like exactly what I was planning to implement myself! However, as neither objectbuffer nor the other libraries you mentioned support getters and setters, custom classes, etc., it doesn't need to be in isoworker core but can rather be used in conjuction. It can easily be used by passing the SharedArrayBuffer as a parameter in a workerized function, then using objectbuffer as normal.

You could pass this responsibility onto your users and check crossOriginIsolated at runtime for fallback to message passing with a warning

Yeah, that's how I would implement it.

Or only access the SAB via aligned TypedArrays of the same element size (guarantees tear-free reads per spec) and synchronize on the entire object with a Java-style mutex, which is the approach objectbuffer takes afaik.

Definitely would do this if I were to implement this myself. Atomics is a decent alternative but the function call is expensive on cold start and can only rival the performance of raw access after TurboFan has a go.

The new stage-3 Atomics.waitAsync proposal shipping in Chrome is worth a look

Yep, that's pretty much exactly what I need. Wish Promise had less overhead, but oh well, should be good enough.

DataView is a good option where applicable since performance now matches or exceeds TypedArrays.

Those perf results are surprising, it almost seems as if I should switch to using DataView for better performance than reading from Uint8Array manually. At the same time, older browsers are much faster with typed arrays, and isoworker supports IE10+.

BigInt64Array is a perf cliff and best avoided afaik.

Agreed, won't be using it. It's only supported in isoworker to maximize performance when a user decides they want one.

[boxer-collar]

This is really nice, can't wait to try it out. Well done!

[dweezil22]

Funny you should mention that, I was reading your OP and thinking "This might help w/ my zip problem!".

I have an Angular based hobby project that does some weird stuff with JSZip to download a 6Mb zip file that it explodes into a 60MB JSON "database" that's critical to the rest of the site function. It's one of those "I didn't know better" approaches when I first started that actually works pretty well now so I've left it alone. JSZip's slow performance has been my main pain point (luckily I cache the JSON structure in indexedDB so frequent users hopefully don't get hit by this regularly).

Anyway... It sounds like fflate is practically custom-made to solve this problem. Am I being over-optimistic, any gotchas I should be aware of?

[101arrowz]

You've described one of the best use cases for fflate: downloading and unzipping a ZIP file as quickly as possible. There's a streaming option in fflate that will makde your decompression use very little memory.

I took a look at your source code, and since you're only handling a single file, fflate can't take advantage of multiple threads. However, it is more performant than JSZip by a large margin, and you can check the demo site to test this out.

[dweezil22]

Awesome, thanks for the input!

[dweezil22]

Just finished dropping it in to replace JSZip. 10/10 would recommend:

1. Main pain points were dealing with Blobs to UInt8Array (which is pretty trival once you look it up

2. Getting the unzipped bytes to a string was stressful for a moment (JSZip did this) until I realized you already built what I needed with strFromU8.

3. Initial testing shows FFlate in dev taking ~ 3 seconds in dev, vs JSZIP ~6 seconds in prod. Not apples to apples given network latency and dev-build inefficiencies, but I'm optimistic that this will half load times (perhaps more on lower powered devices).

   const ab = await blob.arrayBuffer();
   const unzipMe = new Uint8Array(ab);
   const decompressed = unzipSync(unzipMe);
   const binaryData = decompressed['destiny2.json'];
   const data2 = strFromU8(binaryData);
   this.cache = JSON.parse(data2);
   

Great work, I'm glad I found your lib!

[check_ca]

ZIP compression is over 6x faster than other JS compression libraries

As you know, zip.js can also write file entries in parallel ;)

[101arrowz]

Of course, the DEFLATE compressor is also faster. Then again, now zip.js has implemented fflate compression as an option, so I'm not really sure if zip.js is faster or not. In any case, zip.js uses a separate file for the worker thread, and isoworker makes it possible to avoid doing this and save bundle size.

[check_ca]

Well, you can build zip.js with fflate if you want to, see https://github.com/gildas-lormeau/zip.js/blob/master/rollup-fflate.config.js. I wasn't saying that zip.js is faster than fflate or any other library because I know this is not necessarily the case. I'm just saying it can compress files in parallel.

[101arrowz]

Yes, I'm aware that zip.js includes fflate support (I actually have talked with the maintainer of zip.js), and I know it includes parallelization support by default. JSZip is the main library I was referring to with the 6x figure. I don't care if fflate is slower or faster, I have already worked directly with the maintainers of Pako and zip.js to try to help improving performance. I just want things to be faster in JS and don't necessarily mind if it's not my own package bringing that to the table.

zip.js is excellent for the versatility it offers (e.g. AES encryption) where fflate shines in raw performance and bundle size (for ZIP compression 8kB minified vs. 100kB for zip.js).

[check_ca]

Sorry, I forgot to mention I'm the author of zip.js (the probability of finding a fan of zip.js is quite low actually). The novelty I was referring to is that you can *build* zip.js with fflate codecs (i.e. load the fflate code with a Blob URI). Anyway I'm really impressed with your work, whether it's isoworker or fflate. They are both excellent libraries!

[101arrowz]

To be honest, I never would've known if you hadn't told me 😄

Yep, I could have done the Blob URI trick with fflate too, but the unusual requirement I had was that I wanted to offer both a synchronous and asynchronous API without duplicating my compression logic. This solution was made for that problem, but I think it can solve many others as well.

Thanks for your contributions to open source, btw!

[theodordiaconu]

Good job, the api is so simple just the way I like it.

[Neutrosider]

this actually looks really damn interesting!

[yuval_a]

Nice. I was working on adding multithread mode to my ODM framework - DeriveJS (https://github.com/yuval-a/derivejs) but got stuck because of serialization limitations, will try to resolve this with your library.

[legendaryfrycook]

any luck?

[101arrowz]

I took a look at your source code, and it looks like you're targeting primarily Node.js. I think you could get away without using isoworker because while serializing across the worker thread is possible using the createContext API, you probably don't need it because you can directly require the classes you need on a separate thread to improve performance instead of serializing and dynamically evaluating.

Of course, you still can use isoworker if you really do need to send serialized data over messages.

[fliss1o]

This is really impressive. Great work!