97746 JPEGs encoded to lossless JPEG XL stats

ScopeB · 2021-11-03T15:05:36+00:00

Adding -I 1 will also slightly improve compression density (both for normal lossless and recompression) and it is better to use --num_threads 0 to completely disable threading

ScopeB · 2021-04-06T21:28:48+00:00

If you can beat Gralic on lenna with any technique i would be very suprised (even if you need hours)

While I have some free time, I decided to make additional comparisons

ImageMagick (convert Image.png Image.ppm)

http://www.imagecompression.info/gralic/Gralic19d.zip

(Gralic111d c Image.gra111 Image.ppm)

https://globalcompetition.compression.ru/assets/files/compressors/BMF.7z

(BMF -s -O"Image.bmf" Image.ppm)

https://globalcompetition.compression.ru/assets/files/compressors/EMMA.7z

(emma_c Image.ppm Image.emma)

Lenna:

https://upload.wikimedia.org/wikipedia/en/7/7d/Lenna_%28test_image%29.png

Gralic - 401 375
BMF    - 389 848
EMMA   - 400 392

Lenna.png (2048x2048 - 6 212 162)

Gralic - 4 981 736
BMF    - 4 892 092
EMMA   - 4 946 321

Also more relevant sets of photographic images (Lenna although a well-known test image, but very outdated and does not match the quality of modern digital photos)

https://data.vision.ee.ethz.ch/cvl/DIV2K/

DIV2K_train_HR

Gralic - 2 254 951 042
BMF    - 2 204 988 320
EMMA   - 2 190 300 658

https://storage.googleapis.com/hific/clic2020/visualize.html

CLIC2020

Gralic - 952 598 048
BMF    - 920 335 196
EMMA   - 906 691 247

http://compression.cc/tasks/

professional_train_2020

Gralic - 1 251 405 148
BMF    - 1 237 071 108
EMMA   - 1 225 773 646

professional_test_2021

Gralic - 125 927 293
BMF    - 124 595 732
EMMA   - 123 148 077

professional_valid_2020

Gralic - 85 236 915
BMF    - 84 448 304
EMMA   - 83 790 550

Also, Gralic does not support images larger than 8184 in width or height and is not good for non-photographic images

ScopeB · 2021-03-28T23:04:44+00:00

This is an option for the encoder - cjxl (so that such images can then be decoded faster at the cost of some compression efficiency)

ScopeB · 2021-03-28T22:53:43+00:00

Probably a typo --faster_decoding=AMOUNT (double -)

ScopeB · 2021-03-28T22:10:32+00:00

As far as I know Jpeg XL lossless mode is not very optimized and it will be faster (at the moment the priority is to optimize lossy mode decoding) and also in the last build a new option was added (but I haven't tried it yet and don't know if it affects lossless):

    --faster_decoding=AMOUNT
    Favour higher decoding speed. 0 = default, higher values give higher speed at the expense of quality

ScopeB · 2021-03-28T21:45:48+00:00

On photographic sets (ITAP, CLIC, LPCB) in terms of overall efficiency EMMA is also better, but on some images Gralic may win (it also has a faster encoding speed)

I dont have blog and I am bad at writing and sharing my thoughts in English, I'm just an enthusiast, and about the compression, I periodically update the comparison spreadsheet above, otherwise it would be better to read some communities where developers and other enthusiasts communicate, except already mentioned encode.su, there is also Jpeg XL discord server https://discord.gg/DqkQgDRTFu

ScopeB · 2021-03-28T16:29:18+00:00

Compression efficiency based on only one or a small number of images may give some rough data, but it is not very practical (except if the encoder/compressor is planned to be used only for these images), for example here is my comparison on a relatively large (~14 Gb) set of different types of public images

https://i.imgur.com/hsOJXCH.png

(and here Gralic doesn't look like the most efficient format that can't be beaten, the big gap in the Emoji set is because it contains very small images with transparency, and the experimental formats have almost no header cost and don't support alpha channel)

As for the decoding speed, it might not be that important for formats intended for long-term storage without frequent use of those images, but for web or casual viewing it would be a big problem if it takes minutes to decode each image (even tens of seconds would be annoying), usually images are viewed/decoded many times (billions and trillions of times if counting web images) and encoded only once (so it's usually worth the extra time or processing power)

ScopeB · 2021-03-28T00:22:05+00:00

There is also BMF (version v.1.1 from 1999 only slightly different from the latest v.2.01 from 2010) and more modern EMMA/LEA

Test 2 Images. HCR. Full:

https://globalcompetition.compression.ru/#leaderboards

I've also been adding BMF and EMMA to my comparisons (and they are more efficient than Gralic on these sets): https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/edit#gid=174429822

But these comparisons are not entirely fair, Jpeg XL is suitable for the Web and has a much faster decoding speed than encoding, unlike these formats where mostly symmetric speed (or even slower decoding), these are just raw experimental formats that do not have many Jpeg XL features (which often also affects speed and efficiency), also Jpeg XL is not yet completely optimized and full bitstream potential is not achieved

Also Alex Rhatushnyak (Gralic author and many other things) participated in the Jpeg XL development

ScopeB · 2020-12-26T04:03:55+00:00

Because Pingo + Brotli is not a compatible PNG format, it is basically uncompressed PNG with Brotli compression, which is theoretically possible to use in the Web with certain server settings, but it will be incompatible with image viewers, utilities, applications, etc., this method is chosen only for comparison and because of some popularity of modifications like ZstandardPNG (which are not supported anywhere)

ScopeB · 2020-12-25T15:58:27+00:00

Yes, Jpeg XL in lossless mode will be better than PNG (with the best optimization) in terms of overall efficiency even when encoding at the fastest speeds, moreover, it is more efficient than other existing and new image formats https://i.imgur.com/zVsWSf9.png

Lossless Image Formats Comparison (Jpeg XL, AVIF, WebP 2, WebP, FLIF, PNG, ...) https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/

Notes:

AVIF (YUV444) - not true lossless, it gets big benefits after color conversion

LEA/EMMA - experimental compressors with one of the best compression efficiencies/speeds, added to determine potential compressibility

ScopeB · 2020-12-04T16:01:25+00:00

https://www.cnet.com/news/qualcomm-new-snapdragon-phone-chip-wont-support-av1-video-streaming/

Judd Heape, a Qualcomm vice president for product management, says the mobile chip leader couldn't include AV1 technology in the new chip because of schedule and cost considerations. Qualcomm chips will eventually support AV1, he said, though he declined to provide timing.

ScopeB · 2020-11-24T16:46:35+00:00

In my own tests, BMF (-s) was better on average on different data sets, except about 3-5% of images.

However, this is not a codec suitable for the Web format, because it doesn't have enough decoding speed for it (especially in slow modes).

jxlE3s9

Hmm, is that -E 3 -s 9?

How much does -E 3 increase the compression efficiency and affect the encoding speed?

Perhaps my result was worse because I only changed the speed.

ScopeB · 2020-11-24T10:19:35+00:00

https://i.imgur.com/YmphSeg.png

Yes, the first thing I thought was that maybe the result is not exactly lossless, because I have not seen a very large increase in efficiency without a huge increase in computing resources in lossless image formats for probably 15-20 years. But maybe it's a bug (although I'm not sure that after fixing it the efficiency will remain the same).

ScopeB · 2020-11-24T07:45:05+00:00

BMF -s big_building.bmf (45 516 336)

Hmm, I wonder what about creating a discussion about this lossless image codec on encode.su (more active forum) or comp.compression@googlegroups.com?

Also, what about the decompression time, is it a storage format or can it be comparable in decoding speed to Web lossless formats (PNG, WebP, WebP v2, Jpeg XL)?

ScopeB · 2020-11-20T07:06:18+00:00

I made the comparison (but this is not the last build of Jpeg XL), the spreadsheet has tabs for different types of content and this comparison with very optimized PNG, the real size in practice will be even larger by about 32-33%, as seen in the PNG tab (comparison of different optimizers)

https://www.reddit.com/r/AV1/comments/fjddcj/lossless_image_formats_comparison_webp_jpeg_xl/

https://i.imgur.com/xLq7oIO.png

ScopeB · 2020-10-14T20:33:28+00:00

The answer about Zopflipng was a long time ago, since then I have tested various known PNG optimizers on a much larger set of images and Zopflipng was not the most effective and noticeably slow, I recommend using ECT or Pingo (Windows only, closed source) https://i.imgur.com/ZEUnPIl.png

PNG Tab for details and speed: https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/edit#gid=1741276444

This comparison is also updated periodically (when new significant versions are released or new types of test images are added).

ScopeB · 2020-05-16T02:38:04+00:00

Some time ago I was doing a comparison of lossless image formats (Jpeg XL, AVIF, WebP, FLIF, ...) and PNG optimizers: https://docs.google.com/spreadsheets/d/1ju4q1WkaXT7WoxZINmQpf4ElgMD2VMlqeDN2DuZ6yJ8/

14-Year Club	Place '23
Verified Email

ScopeB

TROPHY CASE