/learn

The decoder, step by step

Read in order — each section builds on the one before it. Toggle layers (Concept / Maths / Code) inside any section to control how deep the explanation goes.

1. 01Overview

   What a decoder does. Tokens in, logits out.

2. 02Embeddings

   Token lookup + sinusoidal positional encoding.

3. 03Attention

   Q/K/V, scores, mask, softmax, output. The headline act.

4. 04Feed-forward

   Position-wise W₂·GELU(W₁·x + b) + b.

5. 05LayerNorm & residuals

   Why residuals; why LayerNorm; pre-norm vs post-norm.

6. 06Stacking blocks

   What changes layer-to-layer.

7. 07Sampling

   Logits → next token. Greedy, temperature, top-k, top-p.