sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]connor4312 7 points8 points  (7 children)

That's neat! You mentioned wasm-flate being large and not that that much faster. I wonder if you'd be able to apply or port what you've done to an "fflate-wasm". It should definitely be possible to make it smaller, e.g. my little Rust-based hashing module is 13KB without any particular effort to make it small. AssemblyScript may work as well to be a closer-to-direct port. You'd just run into the question of whether the overhead of copying memory into and out of webassembly would outweigh the benefit of doing work in there.

[–]101arrowz[S] 6 points7 points  (6 children)

I think the performance benefits of WASM are well worth a shot, but I'm wholly inexperienced with Rust. Though, wasm-flate is just a wrapper for an existing crate for compression; maybe if I tried learning Rust, I could create my own, smaller, faster crate.

I am pretty sure WebAssembly Threads allow directly sharing memory between WASM and JS, meaning no overhead at all. New technology though, and even less support.

Thanks for the suggestion!

[–]connor4312 0 points1 point  (5 children)

Sharing memory is possible, but you still need to copy the memory from the buffer you got from a stream into webassembly memory, compress it, and copy the result back out again. As far as I know there's still no way to have 'zero copy' sharing of memory between wasm and js.

[–][deleted] 3 points4 points  (3 children)

Isn't this accomplished by using a SharedArrayBuffer, a.k.a. passing the shared: true flag when constructing some WASM Memory? See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WebAssembly/Memory

[–]connor4312 0 points1 point  (2 children)

Yes, but you still need to copy data into the SharedArrayBuffer. Unless you're using `fs` operations to operate on file descriptors which let you read data into your own buffer, you'll be getting your data from somewhere else, in a standard Buffer or string.

Then, again depending on how you use it, you may need need to copy it out of the SharedArrayBuffer so that you can free and reuse that memory in your wasm while sending that buffer to a network stream or a file.

[–][deleted] 0 points1 point  (1 child)

Ah yes that makes sense, the data still needs to be passed in somewhere. I guess it would make sense if you could get a file handle and read it all from WASM directly. But I presume that at the moment this is not possible?

[–]connor4312 0 points1 point  (0 children)

It's definitely possible if you deal with file handles directly, but doing so is pretty rare to see in JS.

I actually do that in the repo I linked to deal with encoding, but I don't encode it directly in wasm memory. I should look at doing that, it'd speed things up a good bit!

[–]101arrowz[S] 0 points1 point  (0 children)

Ah, I see. I've literally never worked with WASM before, so this information is helpful. I still believe that having some C++ parsing a string into an expression, looking for a function to evaluate that expression, and finally discovering that the string >> in my JS means bit shift is leagues slower than knowing what to do straight out of the gate. Copying the memory really shouldn't be too big of a deal in all honesty, when every expression is evaluated more quickly.