LSTM: Gated Memory So Gradients Can Survive Long Sequences
Hochreiter and Schmidhuber introduced constant-error carousels and multiplicative gates so recurrent nets could learn lags far beyond vanilla RNN horizons. What still applies after transformers, and how to not implement a broken gate.
3 min read