RoPE: Rotary Position Embeddings That Travel Better Than Absolute Vectors
Su et al. rotate query and key pairs by an angle that depends on position, encoding relative offset in the dot product. Why LLaMA-style models adopted it, and what breaks at lengths you did not train.