Before 2017, strong sequence models were mostly recurrent. Attention already existed as an add-on on top of LSTMs (Bahdanau et al., Luong et al.). Vaswani and colleagues made a sharper claim: you can throw the recurrence away and let attention do the whole job of mixing information across positions. The paper is famous because that bet paid, not because it invented the word "attention."
The engineering consequence showed up immediately. Training no longer had a sequential hidden-state chain, so a TPU or GPU could spend the step on large matrix multiplies. Translation quality on WMT, at the time, jumped while training time dropped. Everything since — BERT, GPT, T5, the models behind coding assistants — is a descendant of that training-time argument.
The original stack, as specified
The paper describes an encoder-decoder for machine translation. Six encoder layers, six decoder layers. Each encoder layer is multi-head self-attention plus a position-wise feed-forward network, with residual connections and layer normalization. The decoder adds masked self-attention so position i cannot see the future, and cross-attention into the encoder output.
Scaled dot-product attention is written as:
Attention(Q, K, V) = softmax(QK^T / sqrt(d_k)) VThe sqrt(d_k) term is in the paper for a reason they spell out: without scaling, dot products grow with dimension and softmax saturates. If you reimplement this and "simplify" the scale away, you are not implementing the paper.
Positional encodings were fixed sinusoids in the original, added to embeddings. The authors already noted that learned positional embeddings worked similarly. The industry later moved to relative and rotary schemes because absolute sinusoids get awkward when you want contexts far past the training length.
What did not survive unchanged
Label smoothing, the particular warmup learning-rate schedule, and byte-pair encoding as described are era-specific. Decoder-only language models dropped the encoder entirely. Encoder-only BERT dropped the causal mask. Mixture-of-experts replaced the dense feed-forward block on some layers. FlashAttention changed the IO pattern of the same math.
None of that contradicts the paper's thesis. The thesis was architectural: global mixing via attention is enough, and it parallelizes. Later work is about cost, length, and data, not a return to LSTMs as the default.
How to read it as a practitioner
Read section 3 and table 1, then the ablation in section 5. The ablations are the antidote to mysticism — heads, depth, and key size are treated as knobs with measured BLEU, not as philosophy. When a vendor slide says "we use a transformer," ask which of those knobs they moved and what they measured. That is the same scientific habit the paper is asking for.
If you only remember one sentence from the introduction: replacing recurrence with attention was a systems decision about hardware utilization as much as a modeling decision about long-range dependencies. Both readings are correct, and both still govern why inference is expensive today (attention is still quadratic in naive form) and why GPUs are the default (the remaining work is dense linear algebra).
A worked example
A 2-layer toy transformer: embed, add positional encodings, multi-head attention (softmax(QK^T / sqrt(d_k)) V), residual+norm, FFN. You implement one head, then split d_model across heads. Teacher-forcing a tiny translation pair. You compare a causal mask vs none to see cheating on future tokens.
Count parameters vs an RNN of similar depth on the same task — attention's win was quality at scale and parallelism, not a 10-line demo.
Failure modes
Forgetting the scale sqrt(d_k) (softmax saturates). No mask on decoder self-attention. Mixing up encoder-decoder cross-attention queries vs keys. Positional encoding omitted then wondering why order dies. Treating "attention is all you need" as "no FFN."
Copying 2017 dropout rates into a 2024 LLM recipe.
When this is the wrong tool
Tabular data with 20 features: gradient boosting. Tiny sequences where an LSTM is enough. The paper's architecture is the wrong tool if you need strictly linear time at 1M tokens without approximations. CNNs still win some vision backbones (even if ViTs exist). Do not cite the paper to skip evaluation. RNNs remain fine for tiny on-device models with no GPU.
A worked failure mode
A team reads the 2017 paper, implements multi-head attention in a product ranker, and then trains on 40k rows with no positional scheme beyond "we added an MLP." Offline AUC is fine because the features are mostly bag-of-words already. In production, swapping two clauses in a query ("refund then cancel" vs "cancel then refund") does not change the score. They conclude "attention does not work" and pile on LSTM folklore. The failure is cargo-culting the architecture without the problem it was built for: long-range dependency in sequences where order matters, with enough data and positions. Positional encodings, depth, and compute are not optional decorations. If your task is tabular, a transformer is an expensive kernel; if your task is sequence order, skipping positions is the bug, not the paper.
Attention is the wrong tool when the input is a handful of unordered flags. It is the wrong first model for a 500-row spreadsheet. Do not cite the paper to justify a 12-layer net on a mobile CPU. Use it when you have sequences, positions, and a training budget that can actually move the residual stream.