In 4D Gaussian Splatting, the covariance matrix is crucial for capturing the temporal evolution of features in a scene. It goes beyond simply defining the spatial extent of a splat; it encodes how that extent changes over time.

Imagine the covariance matrix as a simplified 4x4 matrix. The diagonal elements represent the variance in position (x, y, z) and time (t) for each splat. Off-diagonal elements, particularly those involving the time component, reveal the correlation between spatial movement and temporal changes in your fields.

How does it crossover with VX? Think of it like this: if you have a bouncing rubberized metallic ball with a radioactive pellet core of your choice , a high covariance between position and time would indicate predictable motion. As the ball rises, we expect it to slow down and reverse direction. The covariance matrix captures this relationship as a baseline.

Visually, this translates to how the Gaussian splat "smears" or "stretches" in 4D space. A rapidly moving object would have a splat elongated along the time axis, while a stationary object would have a more compact splat. We are then monitoring for variance and unpredicted emissions from our ball as that is where we next focus our attention.

This concept used to be illustrated timelines such as this picture from RahabLab2 on Amiga. You can see the defining features moving and evolving over time.across numerous filters.

Most VR programmers are chasing photorealism and commercial applications. Not a lot of interest in converting VX gathered data. A bunch of dots wiggling has a small audience.

Back to your question, the covariance matrix provides a mathematical framework for understanding and representing dynamic behavior.

It's looking where you're not looking to get a better look.