DeepSeek-R1, read beside the paper

The paper and its explanation side by side.

  • Part 1: The Big Idea: Reasoning from a Reward

    A model that learned to think step by step from one signal: was the final answer right. We read the title, the abstract, Figure 1 and §1, and run the released 1.5B model next to its base model.

  • Part 2: GRPO: Learning from a Group of Answers

    The learning rule behind DeepSeek-R1, term by term: sample a group of answers, score them, and push the model toward the better ones without a critic. We write GRPO from scratch, match it with TRL, and run one real step on a small model.

  • Part 3: Rewards and the Template

    How R1-Zero scores its own answers with two simple rules, why the authors refused a learned reward model, and the one-paragraph prompt that starts training. We write each reward two ways, game a weak reward on purpose, and test two base models in the real template.

  • Part 4: R1-Zero and the Aha Moment

    What pure RL gave DeepSeek-R1-Zero, why its answers grew from hundreds to thousands of tokens, and the moment it wrote Wait. We test the released 1.5B distill.

  • Part 5: The R1 Pipeline: Four Stages

    How R1-Zero's raw skill became DeepSeek-R1: a cold start, reasoning RL with a language reward, rejection sampling into 800k examples, and a last RL stage. We run each idea in Python.

  • Part 6: Distillation: Small Models That Reason

    How 800k answers written by R1 teach six small open models to reason with plain fine-tuning, and why this beat RL on the same model. We write the loss by hand and test a distilled 1.5B model.

  • Part 7: Results, Failures and Limits

    How R1 is scored, where it wins and where it loses, the two ideas that failed, and what R1 still cannot do. We write pass@k and majority voting in Python, and test greedy decoding and few-shot prompts on the 1.5B model.

  • Part 8: Train a Reasoner from Scratch: R1-Zero's Recipe in 330 Lines

    We write the whole DeepSeek-R1-Zero method in a small Python package: the template, the two reward rules and GRPO. We check it against TRL and train a 0.5B model on a laptop.