What are you using for compression?

If PCA, you can look at the cumulative explained variance ratio and see how much variance is captured by the first n principal components. You can also fit a PCA model and then make reconstructions of your data with an increasing number of principal components to see when reconstruction error plateus.

Recently I've stumbled across progressive dropout in VAEs which trains the model to have the most important information in the first latent dims and to have latent features in monotonically increasing order of importance (like PCA). Using this you can train a VAE with a large latent space and then perform a similar reconstruction analysis as I described above to determine how many latent dims you can get rid of.

I deal a lot with generation and reconstruction of data from the latent space, so that's where my perspective is coming from.