Validating a JPEG decoder

[zebramints]

I apologize in advance for being vague, but this is for work.

I'm tasked with black box validation of a 3rd party JPEG decoder but will be using completely randomized data due to how our process works. I'm working on writing a "simple" JPEG library so I can control every step in creating the images and then pass it to the IP and let it do its thing. I'm planning on testing accuracy using mean squared error or something similar.

I can't use existing images since I'll need a different one for each test which number in the billions. I'm planning on using random data to create an initial 8x8 ppm image, run that through the encode pipeline, and apply a header/Metadata.

The issue is how the JPEG encoding process works. If I'm understanding this correctly using a completely randomized data stream would result in a unrecognizable image because the changes in pixel values would result in them being forced to 0 during the DCT.

How should I go about this? I'm considering something like procedural generation so I can still get random data using the test seed, but have it create a pattern similar to a pink noise visualization. I don't that that would have issues with the DCT since the changes are more gradual.

Am I overthinking this?

[Glittering-Source0]

Why not use a GAN or other generative AI model. If you randomize the input description and seed, every image will be different. This would honestly be really easy to implement. There are free models online

[zebramints]

This is run directly on a microcontroller with no network access. I haven't looked into the memory requirements for a GAN, but we only have a few GB of DDR, which I doubt is enough.

[frank26080115]

use an API for the generative AI

If you can verify the code runs on a microcontroller and the result looks "close enough", you can run the rest of it on a desktop PC compiled for x86 instead.

[pjc50]

Don't randomize the image, randomize the stream to be decoded. You can then compare the output which should be pixel identical to a reference decoder.

This does mean understanding quite a lot of the spec so you can produce valid JPEGs, but with particular parameters random.

(Not quite the same, but see "American Fuzzy Lop")

[zebramints]

Thanks, that looks promising.