DeepSeek-R1, explained · Part 1 of 8 · Covers Title, Abstract, Figure 1, §1, §1.1, §1.2

The Big Idea: Reasoning from a Reward

A model that learned to think step by step from one signal: was the final answer right. We read the title, the abstract, Figure 1 and §1, and run the released 1.5B model next to its base model.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, and others. arXiv 2025 (Nature 2025), 2025. arXiv:2501.12948

In January 2025, a lab in Hangzhou released a model that solved competition maths as well as the best closed model of the time. It also published how, and the method was the surprise. Its first model, DeepSeek-R1-Zero, got no supervised fine-tuning on worked reasoning before reinforcement learning. Two rules scored it: was the final answer right, and was the reasoning written inside the right tags. From that alone it found long, careful reasoning by itself. The released model, DeepSeek-R1, then adds thousands of worked examples (the paper's "cold-start data") and more training stages on top.

Each piece below shows the exact lines of the paper, then explains them.

The title

The abstract

These three sentences describe the whole paper. The picture below puts them on one map. Each later part of this series zooms into one box.

The whole paper in one picture1. DeepSeek-R1-Zero (§2.2): reinforcement learning onlyDeepSeek-V3-Basepre-trained, 671BRL with GRPOreward: right answer + formatDeepSeek-R1-ZeroAIME 71.0%Parts 2, 3 and 42. DeepSeek-R1 (§2.3): four stagesV3-Base1. cold startSFT, thousands2. reasoning RL+ language reward3. rejectionsampling + SFT4. RL for allscenariosDeepSeek-R1AIME 79.8%SFT = supervised fine-tuning on examples; RL = reinforcement learning on rewardsPart 53. Distillation (§2.4): small models copy R1800k sampleswritten by R1 + V3SFT on Qwen2.5 and Llamano RL6 distilled models1.5B to 70BPart 6. We run the 1.5B model on this page.
The paper in one picture. Road 1: reinforcement learning straight on the base model gives DeepSeek-R1-Zero. Road 2: four stages, two of SFT and two of RL, give DeepSeek-R1. Road 3: about 800,000 samples from R1 fine-tune six small models. The part of this series that explains each step is under it.

Figure 1

The six benchmarks test different skills. AIME 2024 is the American Invitational Mathematics Examination: 30 hard competition problems, each with a whole-number answer from 0 to 999. MATH-500 is 500 problems from a large set of school competition maths. GPQA Diamond has 198 multiple-choice science questions written by experts so that a web search does not help. MMLU is multiple-choice general knowledge in 57 subjects.

Codeforces is a site for programming contests. SWE-bench Verified asks the model to fix real bugs in real Python projects.

Here is the same figure, redrawn from the paper's numbers, so you can hover each bar:

Figure 1, redrawn from the paper's numbersDeepSeek-R1OpenAI-o1-1217DeepSeek-R1-32B (distilled)OpenAI-o1-miniDeepSeek-V3020406080100AIME 2024, DeepSeek-R1: 79.8AIME 2024, OpenAI-o1-1217: 79.2AIME 2024, DeepSeek-R1-32B (distilled): 72.6AIME 2024, OpenAI-o1-mini: 63.6AIME 2024, DeepSeek-V3: 39.279.8AIME 2024(pass@1)Codeforces, DeepSeek-R1: 96.3Codeforces, OpenAI-o1-1217: 96.6Codeforces, DeepSeek-R1-32B (distilled): 90.6Codeforces, OpenAI-o1-mini: 93.4Codeforces, DeepSeek-V3: 58.796.3Codeforces(percentile)GPQA Diamond, DeepSeek-R1: 71.5GPQA Diamond, OpenAI-o1-1217: 75.7GPQA Diamond, DeepSeek-R1-32B (distilled): 62.1GPQA Diamond, OpenAI-o1-mini: 60.0GPQA Diamond, DeepSeek-V3: 59.171.5GPQA Diamond(pass@1)MATH-500, DeepSeek-R1: 97.3MATH-500, OpenAI-o1-1217: 96.4MATH-500, DeepSeek-R1-32B (distilled): 94.3MATH-500, OpenAI-o1-mini: 90.0MATH-500, DeepSeek-V3: 90.297.3MATH-500(pass@1)MMLU, DeepSeek-R1: 90.8MMLU, OpenAI-o1-1217: 91.8MMLU, DeepSeek-R1-32B (distilled): 87.4MMLU, OpenAI-o1-mini: 85.2MMLU, DeepSeek-V3: 88.590.8MMLU(pass@1)SWE-bench Verified, DeepSeek-R1: 49.2SWE-bench Verified, OpenAI-o1-1217: 48.9SWE-bench Verified, DeepSeek-R1-32B (distilled): 36.8SWE-bench Verified, OpenAI-o1-mini: 41.6SWE-bench Verified, DeepSeek-V3: 42.049.2SWE-bench Verified(resolved)Score in %. The number on each group is DeepSeek-R1. Hover a bar for its value.
Figure 1 of the paper, redrawn from its numbers. R1 and o1-1217 are within about 4 points of each other on every benchmark. DeepSeek-V3, from the same base model, is far behind on AIME 2024 (39.2) and Codeforces (58.7), but close on MMLU (88.5).

The weak group is SWE-bench Verified. R1 fixes 49.2% of the bugs, and V3 already fixed 42.0%. Section 5 of the paper gives the reason. RL on software tasks is slow, because each test of a fix takes a long time. So R1 got little RL on this kind of task.

Introduction

AIME 2024: the share of problems solvedR1-Zero, start of RLR1-Zero, start of RL: 15.6%15.6%R1-Zero, end of RLR1-Zero, end of RL: 71.0%71.0%R1-Zero, majority of 64R1-Zero, majority of 64: 86.7%86.7%o1-0912o1-0912: 74.4%74.4%o1-0912, majority of 64o1-0912, majority of 64: 83.3%83.3%DeepSeek-R1DeepSeek-R1: 79.8%79.8%o1-1217o1-1217: 79.2%79.2%pass@1 unless the row says majority (cons@64). Sources: §1, Table 2, Figure 1.
The AIME 2024 numbers of the introduction next to the models it compares with. Reinforcement learning alone takes R1-Zero from 15.6% to 71.0%, and a vote of 64 answers takes it to 86.7%, above the 83.3% of o1-0912 with the same vote.

Running a distilled model

The smallest student, DeepSeek-R1-Distill-Qwen-1.5B, is small enough for a laptop. Its base model is Qwen2.5-Math-1.5B (§2.4). So we can do something that is hard with the 671B models. We ask the same question to the base model and to the reasoning model, on one machine, and look at every token. The script is r1_part1.py; everything runs on an Apple laptop GPU (mps) in float32.

plain text
== 1. The two models (float32 on mps)
  deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B  params 1,777,088,000  layers 28  d_model 1536  heads 12  vocab 151,936
  Qwen/Qwen2.5-Math-1.5B                     params 1,543,714,304  layers 28  d_model 1536  heads 12  vocab 151,936
  same shapes: True; mean |difference| of one weight matrix (layer 0 up_proj): 0.00406

The two models have the same 28 layers, the same width and the same vocabulary. Every weight matrix has the same shape. The parameter counts differ only because the base model shares one matrix between its input and output layers, and the distilled release stores two copies. The weights themselves moved only a little: in one matrix, the mean change is 0.004. All the difference in behaviour below comes from these small changes, made by SFT on R1's answers.

One decoding step, by hand

A language model writes one token at a time. At each step it gives a score, a logit, to every token of its vocabulary: 151,936 scores here. A decoding rule turns the scores into one chosen token. The paper's evaluation (§3, page 12) samples with "a temperature of 0.6 and a top-pp value of 0.95". We use the same rule everywhere on this page.

Temperature divides every logit by a number TT before the softmax:

pi=exp⁡(zi/T)∑jexp⁡(zj/T)p_i = \frac{\exp(z_i / T)}{\sum_{j} \exp(z_j / T)}

Read it from left to right. ziz_i is the logit of token ii, and pip_i is its probability. The sum in the denominator runs over all tokens, so the probabilities add up to 1. With T=1T = 1 this is the plain softmax. With T=0.6T = 0.6, the gaps between the logits grow by a factor of 1/0.6≈1.671 / 0.6 \approx 1.67. So the top token gets more likely, and the rare ones get rarer.

Top-pp then sorts the tokens by probability and keeps the smallest group at the top whose probabilities add up to at least pp. It throws away the rest and rescales the kept ones so that they add up to 1 again. A long tail of thousands of unlikely tokens can still hold a few percent in total. Top-pp cuts that tail, so the model does not pick a strange token by bad luck.

Here is Method 1, our own Python. Each function does one step:

python
def softmax(z):
    """z: (V,) logits -> (V,) probabilities that add up to 1."""
    e = torch.exp(z - z.max())           # subtract the max so exp never overflows
    return e / e.sum()


def temperature_probs(logits, temperature):
    """Divide the logits by the temperature, then softmax. T < 1 makes the top token more likely."""
    return softmax(logits / temperature)                                  # (V,)


def top_p_filter(p, top_p):
    """Keep the smallest set of top tokens whose probabilities add up to at least top_p; renormalise. p: (V,)."""
    order = torch.argsort(p, descending=True)                             # (V,) token ids, most likely first
    sorted_p = p[order]
    before = torch.cumsum(sorted_p, 0) - sorted_p                         # mass of the tokens ranked above each one
    keep_sorted = before < top_p                                          # keep a token while the set is still short of top_p
    keep = torch.zeros_like(p, dtype=torch.bool)
    keep[order[keep_sorted]] = True
    q = torch.where(keep, p, torch.zeros_like(p))
    return q / q.sum(), int(keep.sum())

The line with before is the one to read twice. For each token in the sorted list, it is the total probability of the tokens ranked above it. A token stays while that total is still below 0.95. So the token that crosses 0.95 stays too, and every token after it goes.

Method 2 is the library. The transformers package has the same two steps as TemperatureLogitsWarper and TopPLogitsWarper, the classes that model.generate uses. We run both on four toy logits, and then on the real logits of the distilled model for its first token:

plain text
== 2. Temperature and top-p on four toy logits
  token   logit   p (T=1)   p (T=0.6)   after top-p 0.95   library
  "18"     2.00     0.609       0.783              0.787     0.787
  "9"      1.00     0.224       0.148              0.149     0.149
  "16"     0.50     0.136       0.064              0.065     0.065
  "2"     -1.00     0.030       0.005              0.000     0.000
  top-p keeps 3 of 4 tokens; max |ours - library| = 6.0e-08

  real logits: the first token the distilled model writes after the GSM8K prompt (vocabulary 151,936)
    'Okay'         p (T=0.6) = 0.7923
    'Alright'      p (T=0.6) = 0.2069
    'First'        p (T=0.6) = 0.0006
    'Hmm'          p (T=0.6) = 0.0001
    'To'           p (T=0.6) = 0.0000
  top-p 0.95 keeps 2 tokens of 151,936; max |ours - library| = 0.0e+00

Check the toy row by hand.

With T=1T = 1, the weight of "18" is e2=7.39e^{2} = 7.39, and the four weights add up to 7.39+2.72+1.65+0.37=12.137.39 + 2.72 + 1.65 + 0.37 = 12.13. So p=7.39/12.13=0.609p = 7.39 / 12.13 = 0.609, as printed. With T=0.6T = 0.6 the same token gets 0.783. Then top-pp adds up 0.783, 0.148 and 0.064. The sum 0.995 is past 0.95, so the fourth token goes, and the three that stay are scaled up a little. Our numbers and the library's agree to 10−710^{-7}.

The real step is more extreme. Of 151,936 tokens, only two survive top-pp: "Okay" and "Alright". The distilled model almost always opens its thinking with one of these two words. It learned this habit from R1's answers.

One decoding step, on four toy tokens1. logits182.091.0160.52-1.02. softmax, T = 1180.60990.224160.13620.0303. softmax, T = 0.6180.78390.148160.06420.0054. top-p 0.95180.78790.149160.06520 (cut)Lower temperature sharpens the distribution; top-p then drops the tail and samples from the rest.
One decoding step on four toy tokens, with the numbers printed by our script. The softmax turns logits into probabilities; temperature 0.6 sharpens them; top-p 0.95 cuts the last token and rescales the other three.

Greedy decoding: our loop against the library

Greedy decoding has no randomness: it always takes the token with the highest logit. So it is a clean test of our own generation loop. If our loop and model.generate both decode greedily, they must write exactly the same tokens.

python
@torch.no_grad()
def our_generate(model, ids, max_new, eos_id, greedy=False, seed=0):
    """Method 1: our own token-by-token loop. The first call reads the whole prompt and fills the KV cache;
    every later call feeds one token and reuses the cache. ids: (1, n) prompt token ids."""
    gen = torch.Generator().manual_seed(seed)
    out = model(input_ids=ids, use_cache=True)                            # prefill: logits (1, n, V)
    past, logits = out.past_key_values, out.logits[0, -1]
    new = []
    for _ in range(max_new):
        tok = pick(logits, gen, greedy)
        new.append(tok)
        if tok == eos_id:
            break
        out = model(input_ids=torch.tensor([[tok]], device=ids.device), past_key_values=past, use_cache=True)
        past, logits = out.past_key_values, out.logits[0, -1]               # (V,) scores for the next token
    return new

The first call reads the whole prompt at once and keeps the keys and values of every layer. This is the KV cache, which LLM Inference, Part 2 explains. Each later call feeds only the newest token. pick is either argmax or our temperature and top-pp functions followed by one random draw.

plain text
== 3. Greedy decoding, 128 new tokens: our loop (KV cache) against model.generate
  our loop 128 tokens, generate 128 tokens, equal at 128 of 128 positions: True
  first 12 tokens: ['Okay', ',', 'Ġso', 'ĠI', 'Ġneed', 'Ġto', 'Ġfigure', 'Ġout', 'Ġhow', 'Ġmuch', 'Ġmoney', 'ĠJanet']

All 128 tokens are the same. (The mark "Ġ" means "a space comes before this token".) With sampling, the two methods write different tokens, even with the same seed. They draw their random numbers in a different order. They draw from the same distribution, which the check in step 2 shows.

The distilled model thinks

Now the real question, the first problem of the GSM8K test set, a set of school word problems. We add the instruction that the official repository recommends for maths, "Please reason step by step, and put your final answer within \boxed{}."

The model's chat format wraps the question in special tokens and ends the prompt with <think>, so the model starts inside its thinking. Our own sampling loop writes at most 2,048 new tokens. Here is the start of what it wrote (the full text is in results/r1_part1_stdout.txt):

plain text
== 4. GSM8K test question 0, distilled model, our own sampling loop (T 0.6, top-p 0.95, seed 0, max 2,048 new tokens)
  | Okay, so Janet has ducks that lay 16 eggs every day. She eats three eggs for breakfast each morning and bakes muffins for her friends with four eggs each day. Then, she sells the remaining eggs at the farmers' market for $2 each. I need to figure out how much money she makes every day from selling these eggs.
  ...
  | So, she eats 3 eggs every morning. There are 24 hours in a day, but since she eats breakfast every morning, which is 12 times a day, right? Because a day has 24 hours, so 24 divided by 2 is 12. So, she eats 3 eggs 12 times a day. Let me calculate how many eggs she eats for breakfast in a day.
  | That would be 3 eggs/morning * 12 mornings/day = 36 eggs.
  | Wait, hold on. That can't be right because she only has 16 eggs a day. If she eats 36 eggs for breakfast, that's way more than she has. That must be a mistake in my reasoning.
  | Let me rethink that. She eats 3 eggs for breakfast every morning. Since a day has 24 hours, but she only eats breakfast once a day. So, she eats 3 eggs once a day. So, that's 3 eggs.
  ...
  ...
  | Then, she sells these 9 eggs at the farmers' market for $2 each. So, she makes 9 * $2 = $18 per day.
  | 
  | Wait, that seems like a lot. Let me confirm each step again.

This is a small model, and it makes a silly mistake: twelve breakfasts a day. But look at what it does next. "Wait, hold on. That can't be right." It checks the result against the problem, 36 eggs eaten from 16 laid, sees the contradiction, and starts again. Nobody wrote this check into the model. It came from R1's answers, and R1 got it from RL. This is the self-verification of §1.1, in a 1.5B model on a laptop.

Then the model keeps checking. It finds 18 dollars early, and then it checks again, and again. The end of its thinking and the start of its answer:

plain text
  | **Final Answer**
  | \boxed{18}
  | </think>
  | 
  | Janet’s ducks lay 16 eggs per day. She eats 3 eggs for breakfast every morning and 4 eggs for muffins every day. The remaining eggs are sold at the farmers' market for $2 each. 
  ...
  new tokens 2048 in 134.3 s (15.2 tokens/s); thinking tokens 1907; answer tokens 140; closed </think>: True
  boxed answer None, gold '18'; correct (our checker) False; correct (math-verify) False

The thinking took 1,907 tokens for a problem that needs two subtractions and one product.

It ended with \boxed{18} inside the thinking. Then the model began a clean answer for the reader. Our limit of 2,048 tokens cut it off after 140 tokens, before its box. So the code marks it wrong. The reasoning was right; the budget was too small. The paper allows 32,768 new tokens for every benchmark (§3).

One answer of the distilled 1.5B model: 2,048 new tokens, cut at the limit1907thinking, inside <think> ... </think>: 1907 tokens140answer: 140 tokensGSM8K test question 0, temperature 0.6, top-p 0.95, seed 0, our own sampling loop. The tag </think> is one more token.
The answer of our own sampling loop, split at the closing think tag: 1,907 thinking tokens, then 140 tokens of the answer before the 2,048-token limit cut it off.

How does the code check an answer? It takes the content of the last \boxed{...} in the answer and compares it with the right answer. Method 1 is our own checker: remove spaces, dollar signs and units, then compare the strings. Method 2 is math-verify, a library that parses both answers as maths, so "0.5" and "\frac{1}{2}" count as equal. A rule that a program can check, with no human and no judge model, is exactly the kind of reward that R1-Zero trained on.

python
def check(pred, gold):
    """Return (ours, math_verify): is the boxed answer equal to the gold answer?"""
    ours = pred is not None and norm(pred) == norm(gold)
    mv = pred is not None and bool(verify(parse('
  

 + gold + '
  

), parse('$\\boxed{' + pred + '}
  

)))
    return ours, mv

Method 2 for generation is model.generate with the same settings: do_sample=True, temperature=0.6, top_p=0.95. Its random draws differ from ours, so it writes a different sample:

plain text
== 5. The same question with model.generate (do_sample, T 0.6, top-p 0.95, seed 0, max 2,048 new tokens)
  new tokens 872; thinking tokens 732; answer tokens 139; closed </think>: True
  boxed answer '18'; correct (our checker) True; correct (math-verify) True

This sample thought for 732 tokens, closed the tag, and gave a boxed 18 in 139 more tokens. Both checkers agree that it is right. Two samples of the same model with the same rule took 872 and more than 2,048 tokens. Thinking length varies a lot from sample to sample.

The base model, same words

Now the same words go to the base model, Qwen2.5-Math-1.5B. A base model has no chat format, so it gets the question and the instruction as plain text. We use our own sampling loop with the same settings and at most 1,024 new tokens.

plain text
== 6. The same question, the same words, base model Qwen2.5-Math-1.5B (no chat format; T 0.6, top-p 0.95, seed 0, max 1,024)
  | To determine how much Janet makes every day at the farmers' market, we need to follow these steps:
  ...
  | 4. The number of eggs remaining to be sold is \(16 - 3 - 4 = 9\) eggs per day.
  | 5. Each egg is sold for $2, so the revenue per day is \(9 \times 2 = 18\) dollars.
  ...
  | The final answer is \(\boxed{18}\). Janet makes $18 every day at the farmers' market.<|endoftext|>
  new tokens 377; stopped by end-of-text: True; contains <think>: False; boxed answers found: ['18']
  first boxed answer '18'; correct (our checker) True; correct (math-verify) True

The base model is right in 377 tokens. It writes a short plan, does the arithmetic, then writes a Python program to check it and prints the program's "output". There is no <think>, no "Wait", and no second attempt.

This is worth a pause, because it is not the naive story. The base model is not a blank page. Qwen2.5-Math was pre-trained and tuned on a great deal of maths, and it already writes short chains of thought. What the distilled model added is the long, self-checking style. On an easy problem, that style costs many tokens and buys nothing.

Pass@1 and majority voting, by code

The last test uses a harder problem from MATH-500: "What is the least positive integer multiple of 30 that can be written with only the digits 0 and 2?" A brute-force loop over the multiples of 30 confirms the answer 2220. Each model writes 8 samples. The distilled model may use up to 6,144 new tokens, and the base model up to 1,024.

plain text
  -- distilled model, max 6,144 new tokens, 701 s for the batch of 8
  sample 0:  6144 tokens, thinking     -, "Wait" x80  answer None         correct False (math-verify False)
  sample 1:  6144 tokens, thinking     -, "Wait" x41  answer None         correct False (math-verify False)
  sample 2:  5197 tokens, thinking  4759, "Wait" x33  answer '2220'       correct True (math-verify True)
  sample 3:  6144 tokens, thinking     -, "Wait" x62  answer None         correct False (math-verify False)
  sample 4:  3465 tokens, thinking  3202, "Wait" x16  answer '2220'       correct True (math-verify True)
  sample 5:  2202 tokens, thinking  1969, "Wait" x5   answer '2220'       correct True (math-verify True)
  sample 6:  6144 tokens, thinking     -, "Wait" x19  answer None         correct False (math-verify False)
  sample 7:  3624 tokens, thinking  3287, "Wait" x17  answer '2220'       correct True (math-verify True)
  pass@1 = 4/8 = 0.500; votes {'2220': 4}; majority answer '2220'; majority correct: True
  -- base model, max 1,024 new tokens, 48 s for the batch of 8
  ...
  pass@1 = 7/8 = 0.875; votes {'2220': 7, '210': 1}; majority answer '2220'; majority correct: True

Every finished sample of the distilled model is right: four of four. The other four were still thinking at 6,144 tokens, with up to 80 "Wait"s. So pass@1 is 18(0+0+1+0+1+1+0+1)=0.5\frac{1}{8}(0+0+1+0+1+1+0+1) = 0.5, the formula of the definition box with k=8k = 8. The vote counts only the samples that gave an answer, and all four say 2220. So the majority answer is right, although only half of the samples were.

The base model is right in 7 of 8 samples, with 196 to 589 tokens each. One sample said 210, a multiple of 30 with a 1 in it. Its vote is 7 to 1 for 2220.

Eight samples per model on one MATH-500 problemdistilled 1.5B: pass@1 = 0.500, majority answer 2220sample 0distilled 1.5B sample 0: 6144 tokens, answer None6,144 tokens, still thinking at the limitsample 1distilled 1.5B sample 1: 6144 tokens, answer None6,144 tokens, still thinking at the limitsample 2distilled 1.5B sample 2: 5197 tokens, answer 22205,197 tokens, answer 2220, rightsample 3distilled 1.5B sample 3: 6144 tokens, answer None6,144 tokens, still thinking at the limitsample 4distilled 1.5B sample 4: 3465 tokens, answer 22203,465 tokens, answer 2220, rightsample 5distilled 1.5B sample 5: 2202 tokens, answer 22202,202 tokens, answer 2220, rightsample 6distilled 1.5B sample 6: 6144 tokens, answer None6,144 tokens, still thinking at the limitsample 7distilled 1.5B sample 7: 3624 tokens, answer 22203,624 tokens, answer 2220, rightbase 1.5B: pass@1 = 0.875, majority answer 2220sample 0base 1.5B sample 0: 410 tokens, answer 2220410 tokens, answer 2220, rightsample 1base 1.5B sample 1: 421 tokens, answer 2220421 tokens, answer 2220, rightsample 2base 1.5B sample 2: 501 tokens, answer 2220501 tokens, answer 2220, rightsample 3base 1.5B sample 3: 486 tokens, answer 210486 tokens, answer 210, wrongsample 4base 1.5B sample 4: 196 tokens, answer 2220196 tokens, answer 2220, rightsample 5base 1.5B sample 5: 461 tokens, answer 2220461 tokens, answer 2220, rightsample 6base 1.5B sample 6: 371 tokens, answer 2220371 tokens, answer 2220, rightsample 7base 1.5B sample 7: 589 tokens, answer 2220589 tokens, answer 2220, rightBar length = new tokens (the scale ends at 6,144). Right answers use the first colour.
The eight samples of each model on one MATH-500 problem, from our run. The distilled model's bars are long: four samples finish with the right answer and four reach the 6,144-token limit. The base model's bars are short, and seven of eight are right.

So, on our laptop and with these limits, the base model did as well or better on both questions. This does not contradict the paper. Table 5 gives the distilled 1.5B model 83.9% on MATH-500, with up to 32,768 tokens per answer. Our 6,144 tokens cut half of its answers.

The run also says something real about this model: the smallest student copies R1's long, careful style, but it checks too often and too long. Test-time scaling is not free. It buys accuracy on hard problems with tokens, and a small model does not always know when to stop. Part 6 compares the distilled models with their base models on full benchmarks, and Part 7 reads the scoring rules in detail.

Contributions

Summary of evaluation results

Next, in Part 2: §2.1 and §2.2.1, the RL algorithm. Part 2 reads Equations 1 to 3 of GRPO term by term, writes the objective from scratch in Python, and checks it against a library.

Run it yourself

The script behind this part is code/papers/deepseek-r1/r1_part1.py. It loads both 1.5B models and runs every step above. The limits are 128 new tokens for the greedy check and 2,048 for the distilled model's GSM8K samples. The base model gets 1,024, and the distilled model's MATH-500 samples get 6,144.

It uses float32, seed 0, and the Apple laptop GPU (mps), or the CPU. The longest step, 8 samples of up to 6,144 tokens, took 701 s. The numbers are saved in results/r1_part1.json.

bash
pip install torch transformers datasets math-verify
python r1_part1.py     # downloads DeepSeek-R1-Distill-Qwen-1.5B and Qwen2.5-Math-1.5B (about 3.5 GB and 3.1 GB)
Terminal output of r1_part1.py: the two models, the sampling table, the greedy check and the start of the distilled model's thinking
The first lines of the output of r1_part1.py, as it ran for this article.

The paper and the code

DeepSeek-R1 on arXiv · official repository · our scripts for this series