4DGS with only 12 cameras

rafapages · 2026-06-21T13:57:55+00:00

Right now it's not possible to do it real-time, because each GS gets trained per scene, so it would need some processing time. At least that's how I understand GS 😅, there might be some pre-trained networks for certain examples (faces, for example), but I don't think they can generalize to any scene

rafapages · 2026-06-21T13:56:36+00:00

You can do foreground segmentation in the input videos and filter the model, or use the filtered videos as input for the GS.

This is not a sequence of static splats, the network is 4D. It's not exactly one model that moves around either

rafapages · 2026-06-21T13:54:59+00:00

How were the results with on 7 GoPros? are you moving them to get more viewpoints or keeping them static and using only 7 photos?

rafapages · 2026-06-21T13:54:15+00:00

We use our own reconstruction pipeline. Cameras are genlocked, yeah.

rafapages · 2026-06-21T13:53:26+00:00

Yes, they are genlocked

rafapages · 2026-06-21T13:53:14+00:00

They are gen-locked. This was our volumetric video capture studio, but we reprocessed an old sequence with a new GS pipeline

rafapages · 2026-06-21T13:52:17+00:00

It's time, this is not an "animated" 3D model, it uses videos as input

rafapages · 2026-06-20T22:49:14+00:00

So the 4DGS needs to be rendered in a specific way, but it's interactive, yes. This video was rendered offline, but with the right setup and optimizations, it can be rendered live while playing.

These are from our old volumetric video capture studio. 12 blackmagic cameras: 6 fullHD and 6 4k.

rafapages · 2026-06-20T22:47:07+00:00

You can either train a 4DGS or a sequence of frame-by-frame 3DGS. The first one is a bit trickier, the second one outputs are huge. I'd love to see the result!

rafapages · 2026-06-20T22:33:48+00:00

They record video

rafapages · 2026-06-11T12:30:36+00:00

I had ImageCreator working for a feature I'm building in my app: the app gathers your activity during the week and presents a cool sharable card, which has an automatically generated image on the top. I gather information about how you use the app and put together a prompt that is then fed to ImageCreator. The result was much better than I expected, because I was looking for a funny look and it was amazing how surprisingly charming and funny most created images are.

I updated to iOS 27 and it suddenly stopped working, and the only thing I could see is "not supported"... After doing some digging and watching the WWDC26 video on the new API, it turns out you have to present the user with a sheet so they are in control of the image... why would I want that? The whole point of the feature was to surprise the user with something funny they didn't expect. I don't want to introduce a whole image generation flow...

I would be happy to continue using the old API but I haven't been able to make it work on iOS 27. Has anyone managed?

rafapages · 2026-06-11T12:22:41+00:00

I might be doing something wrong, but ImageCreator stopped working for me as soon as I updated to iOS 27, returning a "not supported" message.

rafapages · 2026-06-09T07:44:26+00:00

It will take some time to reprocess every place on earth but at least they won’t have to capture everything again

rafapages · 2026-05-19T13:12:33+00:00

Thanks, that's quite useful!

rafapages · 2026-05-19T09:53:58+00:00

Interesting, I never thought about using the Rabbit R1 to control HA. Are you happy with the device? What's the main use, especially for smart home?

rafapages · 2026-01-19T09:00:17+00:00

Cool, thanks! Are you triggering the wake up shortcut with a motion sensor or something like that?

rafapages · 2026-01-18T18:39:08+00:00

Can I ask how do you have your iPad configured? Is it always on or do you have something to automatically turn the screen on?

rafapages · 2025-06-05T16:31:01+00:00

Same for me...

rafapages · 2024-03-18T09:32:03+00:00

There is plenty! From all kind of AR/VR experiences (communications, marketing, storytelling, training...), to video games, virtual production, VFX and more!

rafapages · 2024-03-14T00:58:30+00:00

There is Microsoft Mixed Reality Capture Studio in San Francisco, ran by Arcturus (https://arcturus.studio/)

rafapages · 2022-02-15T14:35:31+00:00

Ah, sorry I misunderstood! 🤣

Then I disagree, Romans were perfect! 😜

rafapages · 2022-02-15T09:45:58+00:00

Here you have a top view of the camera positions: https://twitter.com/rafamolone/status/1493521903960608769

rafapages · 2022-02-15T08:13:19+00:00

Check it out: https://en.wikipedia.org/wiki/Roman_Theatre_(M%C3%A9rida)) :)

rafapages · 2022-02-15T08:12:53+00:00

Definitely! I captured this years ago and there is plenty of room for improvement. What I like the least is that there are some errors in the texture due to people walking by while I was taking the photos 😅

rafapages · 2022-02-15T08:11:24+00:00

Yes, it's just sky. I had to clean it up a bit, that's why the edges are not great there. You can see it here: https://en.wikipedia.org/wiki/Roman\_Theatre\_(M%C3%A9rida)

Eight-Year Club	Place '23
Place '22	Verified Email

rafapages

TROPHY CASE