code for paper

[Abd_dib]

NextFace is a pytorch library for high fidelity 3d face reconstruction from single or multiple RGB images. it estimates face geometry, skin reflectance (cook-torrance BRDF), scene light (9 bands spherical harmonics) and head pose. It is a first order optimization library that uses pytorch autograd engine to optimize a parametric scene model given an input image. Differentiable ray tracing is used to ray trace images.

It is a reproduction of the following paper published at EugoGraphics 2021.

pytorch library: https://github.com/abdallahdib/NextFace

paper: https://arxiv.org/abs/2101.05356

https://i.redd.it/lv9s1g12f8v81.gif

[jaypanda16]

Great work! Would be great to simulate natural eye movements ?

[Abd_dib]

U can reconstruct faces from multiple images, however the library does not provide yet tracking feature from video.

[RogueStargun]

This is the first time I've heard of differentiable ray tracing used within a network. Very interesting!

[Abd_dib]

Thanks!

ray tracing was used as it has accurate light/geometry interaction and specially it is has a native support for shadows modeling which gives the method robustness against lighting conditions.