Mixed-Precision Training: FP16 Math With FP32 Where the Bits Actually Matter
Micikevicius et al. (2018) trained deep nets with half-precision arithmetic using loss scaling and fp32 master weights. The recipe still sitting under AMP, bf16, and most large-model trainers.
3 min read