Kaplan Scaling Laws: Loss as a Power Law in Size, Data, and Compute
Kaplan et al. 2020 fitted smooth power laws for transformer language-model loss. What the laws still help you plan, and which extrapolation the Chinchilla generation had to unwind.
2 min read