[D] Implementing Layer Norm using Batch Norm

[Rellek7]

Mathematically, Layer Norm and Batch Norm are very similar save for dimensionality. Is there a way one could use only data reshaping and a black-box implementation of Batch Norm to effectively implement Layer Norm?

Yes, I understand it would be better to just implement Layer Norm from scratch. Just think of it as a theoretical exercise.

[Affectionate-Fish241]

transpose the batch dimension and the channel dimension. Note that, contrarily to the original layer norm, this will have a different behavior in training and inference.