PagedAttention: Treat the KV Cache Like a Virtual-Memory Problem
Kwon et al. (vLLM) allocate KV cache in pages, remap on the fly, and stop wasting giant contiguous buffers on every sequence. Throughput, sharing for prefixes, and the operational model of an LLM OS.
3 min read