Batch Normalization: Fix the Covariate Shift Story, Keep the Engineering Win
Ioffe and Szegedy normalize activations using minibatch statistics so deep nets train faster and tolerate higher learning rates. Population stats at test time, and why batch size became a hyperparameter of the layer.
3 min read