Circuits, read beside the paper

The paper and its explanation side by side.

• Part 1: The Big Idea and the Residual Stream

  The opening of A Mathematical Framework for Transformer Circuits, read slowly: what mechanistic interpretability is, why the paper studies tiny attention-only models, the whole model written as four equations with every symbol and shape, the residual stream as a shared channel that layers read from and write to, virtual weights, the bandwidth problem, and why a zero-layer transformer can only learn bigrams. Every equation derived by hand, then checked on real models.

• Part 2: One Attention Head, Taken Apart: the QK and OV Circuits

  The attention-head sections of A Mathematical Framework for Transformer Circuits, read slowly: why heads are independent and simply add, how a head moves information between tokens, the tensor-product form h(x) = (A ⊗ W_O W_V) x explained with a tiny example, why the four matrices of a head collapse into two low-rank ones, and the QK and OV circuits read straight from the weights of a real model.

• Part 3: Paths Through the Model: the Path Expansion Trick

  How the paper turns a one-layer model into a sum of end-to-end paths, one for the token itself and one for each head. We expand the real model, freeze its attention, and check every term.

• Part 4: One-Layer Models: Skip-Trigrams, Copying and Their Bugs

  What a one-layer attention-only model really learns, read from its weights: skip-trigrams, copying, heads that only look at position, the bugs of a factored form, and eigenvalues as a test for copying.

• Part 5: Two-Layer Models: How Heads Compose

  With a second layer, one head can read what an earlier head wrote. We follow the paper's three kinds of composition, expand the logits and the attention scores into paths, and measure which paths matter in a real two-layer model.

• Part 6: Induction Heads

  How a two-layer model learns to continue a pattern it saw earlier in the text. We find the two heads that do it in a real model, test them on random tokens, and break them on purpose.

• Part 7: What the Paper Left Out, and What Came After

  The paper's own view of its limits, the MLP layers it could not read, and the later papers that grew from it. We train a tiny model that packs five features into two numbers, and read real MLP neurons.