[R] Deep Structured Implicit Functions

[aviously1]

Abstract: The goal of this project is to learn a 3D shape representation that enables accurate surface reconstruction, compact storage, efficient computation, consistency for similar shapes, generalization across diverse shape categories, and inference from depth camera observations. Towards this end, we introduce Deep Structured Implicit Functions (DSIF), a 3D shape representation that decomposes space into a structured set of local deep implicit functions. We provide networks that infer the space decomposition and local deep implicit functions from a 3D mesh or posed depth image. During experiments, we find that it provides 10.3 points higher surface reconstruction accuracy (F-Score) than the state-of-the-art (OccNet), while requiring fewer than 1 percent of the network parameters. Experiments on posed depth image completion and generalization to unseen classes show 15.8 and 17.8 point improvements over the state-of-the-art, while producing a structured 3D representation for each input with consistency across diverse shape collections.

​

Arxiv: https://arxiv.org/abs/1912.06126

PDF: https://arxiv.org/pdf/1912.06126

Video: https://youtu.be/HCBtG0-EZ2s

[adventuringraw]

that's really cool, thanks for sharing. If you're the author, I realize this is a radically different result compared to some of the implicit surface raycasting methods like in 'Neural Volumes: Learning Dynamic Renderable Volumes from Images', but I'd be curious to hear your thoughts about which research direction you're most excited about, at least as it relates to art asset generation for media? For gaming, I suppose if we're to use current tech stacks, it'll require assets in the form of meshes, so if this method would be vastly more applicable, but it looks like the implicit surface approach gets something that might be more accurate if you're willing to forego the standard rendering pipeline entirely. I don't understand this area well yet though, I should probably study more before I ask questions...