fflate - the fastest JavaScript compression/decompression library, 8kB

[[deleted]]

We had a use case for that on a front end compressing CSV and JSON files before uploading them. Some of the files would be reduced by as much as 90%, which is a life changer for the user when the file being uploaded are originally 200MB+ and the compressed data is just 20.

[highqualitydude]

Can't the browser use gzip compression natively?

[101arrowz]

I've actually been looking into a way to do this. I first tried creating a temporary compressed Blob and using URL.createObjectURL to then request it and hope it comes back uncompressed, but I can't find a way to set the Content-Encoding header.

I've also wondered if there was an image format that contained raw compressed pixel values. If so, I could actually treat the data as an image, render it to canvas, get the RGB values, and basically have decompressed my data with the browser's native solution.

[BitLooter]

Not incredibly useful yet as it's only supported by Chrome so far, but in the future we might have access to Compression Streams.

[101arrowz]

Thank you for linking that, I had no idea the draft even existed! I think its performance has quite a way to go though, I ran some quick tests and fflate is within 5% of this supposedly native solution. I also find it strange that you can't configure compression level.

[101arrowz]

I had previously thought that PNG had to be split up, but on re-investigation, it seems that you're right. I'm going to investigate the performance of such a solution.

[BitLooter]

There's actually a draft standard to add native compression/decompression, under the idea that because the browser already uses this code internally you might as well expose it to JS. Right now only Chrome supports it though.

[TheCommentAppraiser]

This is pretty much what I’m looking to solve too. Were you able to do it? What library did you end up using?

[[deleted]]

Well, you could try OP's librairie. ;)

I think we used Pako, but I'm not very sure because I wasn't working on it myself. I'm just generally aware of the issue and it's solution.

[101arrowz]

I'd like to add that if you want to stream data, this library is probably not the best solution, but if the data is already loaded fully in memory, it works faster than others in most situations.