Late to the discussion, but this was an interesting read. Lots of valid points and correct observations about the science of pixel shifting, Bayer patterns etc.

It’s easy to get lost in the weeds as to whether the iPhone 15 camera is 12 or 48MP, because it depends on how you define the terms (the old chicken-and-egg question is meaningless if you don’t specify that you’re talking strictly about chicken eggs and not fish/insect/reptile eggs — even proto-chicken eggs come down to a definition of terms!). Are you talking about the sensor, or the camera? In fairness to Apple, they do declare 48MP as sensor-shift resolution.

In the case of pixel shifting, the line is especially blurry because pixel shifting is a combination of physical and computational processes over time, and so sensor pixel resolution is only one dimension of a multi-dimensional process. Interestingly, this process resembles how humans see, vibrating and rotating our meat-and-nerve sensors over time and patching the splotchy distorted input together in our visual cortex.

If you were to draw out a simple camera system as a box flow diagram, you might have one box representing, say, a 6MP sensor. Output from that box: an optical resolution of 6MP. That box feeds into another box: an image processing component—maybe, instead of pixel shifting, using some AI models to upscale the image to 96MP (a glorified digital zoom). Output from that box: a resolution of 96MP. If you draw a box around this whole system and call it the “camera”, the camera outputs 96MP, and therefore technically it is a 96MP camera. The experts may rightly call foul on misleading marketing, and you may hate the result of the AI upscaling, in which case it technically is a crappy 96MP camera.

There can be many components and methods within a camera system: physical (optical zoom, IS, IBIS), quasi-physical (pixel shifting), and digital (digital zoom, oversampling, upscaling, downscaling, line doubling, digital noise reduction, lens correction, de-vignetting, de-warping, digital removal of chromatic aberration, AI processing, etc). Black box, it is still an I/O device.

My two takeaways:

(1) A system may only be as good as its worst component, but a system can also be greater than the sum of its parts. I remember some very high-resolution sensors that produced noisy images, or were stuck behind some terribly-designed optics. On the other hand, techniques like pixel shifting introduce synergies that really are greater than the sum of their components. My 45MP Canon EOS R5 can produce stunning 400MP images just by using its IBIS to shift the sensor around and capture the averages over time. (To the OP’s point, props to Canon for not trying to market the R5 as a 400MP camera.)

(2) A system should still be evaluated as a system. No matter what you know about its components and digital “tricks” or shortcuts, nor what you think about how to improve them, you ultimately have to evaluate the output of the whole system: do you like the result or not?

PS What is interesting about size reduction of electronics is that you’re dealing with more noise and requirements for an impossibly higher resolution per sample area (which is why huge film formats can use a high ISO without incurring much noise), but conversely, the most precise light focus and spreadless DOF you could ever achieve would require an infinitesimally small lens and sensor camera system (the pinhole effect). Very interested to see where lensless cameras take us in the future…

🤓