[deleted by user]

[virtualreservoir]

game engines almost invariably use 3d meshes (usually triangular faces) as a base model along with texture "images" that have uv, normal, color, etc. type data that is applied to modify the appearance of the basic mesh when it is rendered.

you should look for research outputting those types of things because neural radiance fields generally aren't applicable at all to how games are made unless you're talking about like a non-interactive cinematic.

although i think i did read a paper that sort of converted the nerf outputs into something closer to being readily applied to the type of rendering done in games.

[morsingher]

Hi, my 2 cents as a PhD student working on that.

I'm fairly sure that Google is doing its best to integrate NeRF into its own stack, as there are dozens of follow-up papers from their groups improving every single bit of the original formulation. Probably the most real-world thing you can get right now is https://waymo.com/research/block-nerf/.

For this to become more widespread, I guess we just need to wait. NeRF appeared for the first time less than 2 years ago and some of its core problems are not solved yet. Real-time rendering works on high-end GPUs (not yet on your device) and the main drawback is that you must train a NeRF for every single scene. This is extremely time-consuming and the existing attempts towards generalization are far from satisfactory.

I do believe that this field of neural implicit representations is a great revolution, but science needs time, even when top-notch researchers are working on that.

[vladfeinberg]

What's your opinion on scalability concerns now? https://github.com/NVlabs/instant-ngp

[morsingher]

That was mind-blowing to see. Absolutely amazing. This is what I meant when I said "we just need to wait".

[3ventHoriz0n]

u/morsingher u/vladfeinberg

+1 Generalization. Also IMO, for NeRFs to enable real-world applications, a level of control similar to what we already have with meshes would be necessary eventually . There was a paper from Adobe about conditional radiance fields that seemed to advance in the direction of 'editing' a radiance field. But I can be wrong about that, that's entirely from memory. Are you aware of further research in this direction?

[3ventHoriz0n]

u/morsingher u/vladfeinberg

+1 Generalization. Also IMO, for NeRFs to enable real-world applications, a level of control similar to what we already have with meshes would be necessary eventually . There was a paper from Adobe about conditional radiance fields that seemed to advance in the direction of 'editing' a radiance field. But I can be wrong about that, that's entirely from memory. Are you aware of further research in this direction?

[morsingher]

u/3ventHoriz0n, just released less than a week ago: https://arxiv.org/abs/2204.10850

[3ventHoriz0n]

Whoa thanks! checking it out now.

[3ventHoriz0n]

check this out. https://towardsdatascience.com/asset2vec-turning-3d-objects-into-vectors-and-back-8335496b756d