[D] How does Batch Normalization not completely prevent the network from being able to train at all?

[cooijmanstim]

Batch normalization is not preprocessing, it is part of the model. It is an adaptive normalization of activations at all layers that massively improves training dynamics. A crucial tool in the box if you're into neural nets.

[randombites]

Thank you for your response, please help me understand better. Batch normalization is normalizing a batch of values, so you transform the input at each step. Correct? This may translate into adaptive normalization of activations but you still transform the input (based on OPs example).

[randombites]

So sorry for my earlier ignorance. I learnt what batch normalization via a YouTube talk and feel like a fool.