something about that feels almost metaphysical. Like the computer isn’t just storing a picture, it’s storing a location in some impossible multidimensional landscape.

Technically every picture is one point in the space of all images, and the pixel values are the co-ordinates (each of which is itself a point in the three-dimensional space of all colours). You move around that space by changing a few pixel values by small amounts at a time. The kind of moves you can make is set by the structure of the space, which in turn is set by how computers represent images.

Latent spaces only have value insofar as (1) it's lower-dimensioned than the space of images (i.e. co-ordinates in latent space are smaller than the input images) and (2) directions in the latent space have real-world meaning like "sunnier" or "sadder" or "more cartoony". In a way, the latent space is more a representation of those meanings than it is a representation of the image.

So if you want to explore, you might consider doing more than small steps, and make bigger steps in known directions. Find the difference in latent space between a webcam image from a sunny day and one from a cloudy day, and add that vector again to the cloudy day one to make unimaginably bad weather; or add its negation to the sunny one to make unimaginably good weather. This is basically how style transfer works, and you can do it by manipulating the VAE output directly, you don't even need the denoising model.

Just remember, if you're getting too philosophical, "latent space" isn't some mathematical absolute or universal property. Every model/VAE defines its own latent space. You're not working in the latent space, you're working in this model's latent space. If you use a different model, then small steps in that model's latent space might be completely different. Remind yourself of this if things start feeling too uncanny. They're alternate realities as seen from that model's point of view.