AI FlashAttention: Why IO-Aware Kernels Matter More Than Another 2% BLEU Dao et al. showed that attention was memory-bound, not math-bound. Here is the IO story and what it changed for long-context training. Jul 28, 20264 min read Read →