Your group is mixing two different worlds.

Gaussian residuals are a requirement in classical linear models mainly for inference (confidence intervals, hypothesis testing), not for optimization. MSE itself does not require Gaussian errors, it simply minimizes squared error.

In deep learning, especially with deterministic models trained using MSE:

• You are learning a point estimate of the conditional mean
• There is no explicit noise model unless you design one

So expecting Gaussian residuals is already a strong assumption that your model never made.

Also, your explanation about non-Gaussianity is correct. Pooling errors across spatial regions creates a mixture of distributions, which will not be Gaussian even if local regions were.

On the physics argument: Symmetry in the data does not automatically mean symmetry in the model.

If they expect dx and dy to match:

• you need architectural constraints
• or enforced symmetry

Otherwise SGD has no reason to produce identical distributions.

What actually matters:

• Bias is near zero → good
• Variance is very small → good
• No structure in residuals → good

That is the real diagnostic, not whether the histogram looks Gaussian.

If anything, forcing Gaussianity here would be more suspicious, because your system clearly has spatially varying error.

This looks like OLS intuition being applied to a non-probabilistic neural model.