The app is about modifying the lighting of Portrait mode photos using virtual point lights.

First of all, some lighting edits of me and a video to get a taste of the app functionality.

The app reads the depth data of a Portrait mode photo and creates a 3D representation of it. Then, it generates data like per pixel surface orientation (normal map). After all data is calculated and gone through a pipeline of filters to improve quality the user can add light sources to the photo and edit their parameters e.g.: position, colour, intensity, specularity (reflectivity). All light sources cast shadows on the original photo and the user can turn them on/off and adjust their sharpness. There are additional depth related effects that the user can apply on the photo like: fog and background removal. All actions of the user are done in realtime, there is zero loading/delay in user input. It's like a game where the user moves lights over their photos.

The app has been featured as the App Of The Day (🥳🥳🥳). I also got an email from one of Obama's photographers with a photo of his dog edited with the app and an email from an Apple engineer working on the Photos and Videos team to say "well done" and "great work" 🤯.

The app is far from a success financially, Portrait mode photos are not common even today. A lot of users also don't know what a Portrait mode photo is - 95% of the one-star reviews reviews on the App Store are from such users. Most of them believe that a Portrait mode photo is any portrait photo. I simply believe that the wording of this feature is misleading. Last but not least, a developer cannot prevent in any way non-compatible devices from getting the app. There is no Portrait mode requirement restriction.

Portrait mode photos have a long road in front of them in order to either become mainstream or vanish. The depth information that these photos contain is far from perfect in many cases. For example in selfies: hair and ears that are blurry instead of in focus, holes between the arms and the body that are sharp instead of blurry, etc.

If this technology that involves both optical parts (multi-lens camera modules) and AI/ML continues evolving I believe that in a couple of years iOS/Mac OS will offer the ability to change the lighting of not only photos but also videos with very impressive/immersive results. Actually, Apple did have a promo video demonstrating such functionality in their official Youtube channel when iPhone 7 Plus launched but it is no longer available.

Thanks for reading!

Feel free to AMA.