How DeepSeek stopped each new token from reading the whole conversation: NSA, the token scorer of V3.2, its measured costs and the price cut.
On 29 September 2025, DeepSeek released a model called V3.2-Exp and announced "API prices cut by 50%+". Output tokens went from $1.68 to $0.42 per million. The experts were the same as before, and so was the size: 671 billion parameters.
The only change to the architecture was in one place: how each new token looks back at the tokens before it.
That change is called DeepSeek Sparse Attention, or DSA. It came out of a research paper from February 2025, Native Sparse Attention (NSA). Both answer one question. When a model has read 128,000 tokens, which of them does the next token really need to look at? DeepSeek shipped its answer together with a 75% cut in the output price.
This part builds on two earlier pages. Attention, Part 3 covers sliding windows and a first look at DSA. Part 2 of this series covers the MLA cache. Short reminders appear where they are needed.
A model that answers a question works in two phases. First it reads the whole prompt in one pass. Then it writes the answer, one token at a time. The two phases have very different costs. The first is prefill, the second is decode (LLM Inference, Part 1 explains both).
In attention, each token makes a query. It compares the query with the key of every earlier token. The comparison gives a score. A softmax turns the scores into weights that add up to 1. The weights then mix the values of the earlier tokens into one output. So the work grows with the number of (query, key) pairs.
Count the pairs in a prompt of 4 tokens. Token 1 looks at 1 token (itself), token 2 at 2, token 3 at 3, token 4 at 4. That is 1 + 2 + 3 + 4 = 10 pairs. For a prompt of L tokens the count is:
P(L)=1+2+⋯+L=2L(L+1)
Here P(L) is the number of (query, key) pairs and L is the prompt length. For L=4 it gives 4×5/2=10, as counted. For L=131,072 (128K) it gives about 8.6 billion pairs, in every layer. Double the prompt and the pairs grow four times. This is the quadratic cost of prefill.
Decode looks cheaper at first. One new token at position L makes one query and compares it with L keys. That is linear in L, not quadratic. But it happens for every token of the answer, and each comparison needs the earlier tokens' records from memory.
Now put in V3's real sizes. V3 uses Multi-head Latent Attention (MLA, see Part 2). Instead of full keys and values per head, MLA stores one small latent vector per token per layer. It has 512 numbers, plus a 64-number key part that carries position (the RoPE key). All 128 query heads read that same 576-number entry.
MLA can run in two forms, and the DeepSeek-V3.2 paper draws them side by side.
Per (query, key) pair and per layer, the two forms cost these multiply-adds:
Read the first one left to right. There are 128 heads. Each head takes a 576-number dot product for the score, then adds a 512-number latent, scaled by its weight, into its output. The second one is the same with 192-number keys and 128-number values.
For one new token at position L, the decode cost of attention over all 61 layers is then:
FLOPs per new token=2×61×139,264×L
The 2 turns multiply-adds into FLOPs. At L=131,072 this is about 2.2 trillion FLOPs. Compare it with the rest of the model. V3 activates about 37 billion parameters per token, and each costs about 2 FLOPs, so the experts and projections cost about 74 billion FLOPs. Our calculator (part5_cost.py) prints both:
plain text
DECODE: one new token at position L, whole model (61 layers), attention core only L dense GFLOP DSA idx DSA main DSA tot dense/DSA weights attn/weights 8,192 139.2 8.2 34.8 43.0 3.2x 74 1.9x 16,384 278.4 16.4 34.8 51.2 5.4x 74 3.8x 32,768 556.7 32.7 34.8 67.5 8.2x 74 7.5x 65,536 1,113.5 65.5 34.8 100.3 11.1x 74 15.0x 131,072 2,226.9 131.0 34.8 165.8 13.4x 74 30.1xDense attention FLOPs equal the 2 x 37B weight FLOPs at L = 4,355 tokens
Look at the last column. At 8K tokens of context, dense attention already costs 1.9 times the whole rest of the model. At 128K it costs 30 times more. The mixture of experts made V3 cheap per token, but at long context attention eats that saving many times over. Above about 4,400 tokens, attention is the bigger bill.
Attention FLOPs for one new token of DeepSeek-V3, by context length (our calculator, attention core only). Dense MLA grows in a straight line with the context; DSA grows much more slowly; the expert and projection FLOPs stay at about 74 billion.
FLOPs are only half of the decode story. Each new token must also read the cache of every earlier token from memory. With the FP8 layout of DeepSeek's FlashMLA kernels, one token's entry is 656 bytes per layer. With plain BF16 it is 1,152 bytes.
plain text
DECODE: cache bytes read for one new token of one sequence (61 layers) L dense FP8 MB dense BF16 MB DSA idx MB DSA main MB DSA tot MB dense/DSA 8,192 327.8 575.7 66.0 82.0 147.9 2.2x 16,384 655.6 1,151.3 131.9 82.0 213.9 3.1x 32,768 1,311.2 2,302.7 263.8 82.0 345.8 3.8x 65,536 2,622.5 4,605.3 527.7 82.0 609.6 4.3x 131,072 5,245.0 9,210.7 1,055.4 82.0 1,137.3 4.6xAt 128K on one H200 (4.8 TB/s): reading the dense FP8 cache takes 1.09 ms per new token, the DSA reads 0.24 ms (one sequence, before any batching)
At 128K, one new token of one conversation reads 5.2 GB of cache. On an H200, whose memory delivers 4.8 TB per second, that is about 1.1 ms of pure reading per token. A server decodes many conversations in one step, and each one brings its own cache. The weights are read once per step and shared by the whole batch, but the caches are not shared. So at long context the cache reads, not the weights, set the speed.
Cache bytes read for one new token of one sequence, by context length (our calculator). The dense lines grow in proportion to the context. DSA still grows, because the indexer reads a small key for every token, but about 4.6 times more slowly at 128K.
The way out: most of the weight lands on a few tokens#
Attention weights are not spread evenly. In a long document, a token puts most of its weight on a few places. These are often the last few words, a name mentioned earlier, or the question at the top. The other tokens get tiny weights. If the model knew in advance which tokens matter, it could skip the rest.
The hard part is the "in advance". To know which tokens get large weights, you normally have to compute all the scores, and that is the cost you wanted to avoid. Each method below is an answer to this puzzle. The picture shows the patterns on 16 tokens.
Which (query, key) pairs get computed in four kinds of attention on 16 tokens. Dense computes every pair. A sliding window keeps the last few tokens. NSA's selection branch picks whole blocks. DSA picks single tokens anywhere.
A sliding window (second panel, see Attention, Part 3) is cheap, but it forgets anything older than the window. DeepSeek wanted to reach a fact 100,000 tokens back, so it needed selection that depends on the content.
There is a second trap. A model trained with full attention learns to rely on all its tokens. Cut most of them away after training, and quality drops.
NSA is a research paper by DeepSeek-AI and Peking University, posted on 16 February 2025 and published at ACL 2025. No released DeepSeek model used NSA exactly as published. But its ideas are the direct parents of DSA, and of the compressed attention in V4.
NSA in one picture. The past is cut into blocks. Compression turns each block into one summary key. Selection reads a few whole blocks in full. A sliding window keeps the most recent tokens. Learned gates mix the three outputs.
The compression branch does more than give a coarse view. Its attention scores already say which parts of the past matter. NSA reuses them as importance scores for selection blocks of 64 tokens. It then reads the top 16 blocks in full. So the scorer is free: it is a by-product of the compression branch.
Why whole blocks and not single tokens? The answer is the hardware.
A second hardware rule decides who chooses the blocks. When heads share one cache, as in GQA or MQA, NSA adds up the importance scores of all heads in a group. So the whole group picks the same blocks. If 16 heads each picked their own 16 blocks, the group could load 256 blocks, and the saving would be gone. DSA takes this rule to the end: in V3.2, all 128 heads of a layer share one selection.
From the settings, the number of keys one query reads at position t is:
Nt=dt+nl′+w
Read it left to right. There are t/d compressed keys, one per stride of 16 tokens. There are nl′ selected keys, 16 blocks of 64. And there are w window keys, 512 of them. At t=65,536:
Nt=1665,536+16×64+512=4,096+1,024+512=5,632
Full attention reads 65,536 keys at that position, so NSA reads 11.6 times fewer. The exact count of compressed blocks is ⌊(t−l)/d⌋+1=4,095, one fewer. The paper rounds to t/d, and our calculator prints both. Table 4 of the paper gives the same numbers.
Keys read per decoding step by full attention and by NSA, at 8K to 64K tokens (our calculator with the paper's settings; the same numbers as the paper's Table 4).
The training kernels were timed for real.
And quality? The test model was a 27-billion-parameter mixture of experts with 3 billion active parameters, trained from scratch. NSA matched or beat the same model with full attention. It scored higher on 7 of 9 general benchmarks and got perfect retrieval in a 64K needle-in-a-haystack test. On the LongBench long-context suite it averaged 0.469 against 0.437 for full attention.
These are the authors' own results on their own model. They show that native training works at that scale, not that NSA beats full attention in general.
From NSA to DSA: what DeepSeek kept and what it dropped#
Seven months later, DSA appeared in DeepSeek-V3.2-Exp. It is not NSA with new numbers. It keeps one idea, drops two, and adds one.
NSA (Feb 2025)
DSA (Sep 2025)
What is selected
whole blocks of 64 tokens
single tokens
Who scores
the compression branch, for free
a separate lightning indexer
Branches
compression + selection + window, gated
one: attention over the top-k
Shared selection
per group of heads
all 128 heads of a layer
How trained
from scratch
continued training of an existing model
Tested on
27B research model
the 671B production model
The biggest change is the last row. DeepSeek did not want to train a new 671B model from scratch to try sparse attention. It wanted to add sparsity to V3.1-Terminus, a model already trained with dense MLA. That decided the design. The new parts had to fit around MLA without changing it, and they had to learn in continued training.
Where DSA sits in one of DeepSeek-V3.2's 61 layers. The lightning indexer and top-k selector attach to the MLA attention and tell it which cached entries to read. Everything else is V3.1-Terminus.
The new weights are small. By our count from the V3.2-Exp reference code, the indexer adds about 14 million weights per layer, about 0.85 billion over 61 layers. That is 0.13% on top of 671 billion. DSA has two parts: the lightning indexer, and the top-k selection that uses its scores.
It,s is the score that query token t gives earlier token s.
HI is the number of indexer heads (64 in V3.2), and j counts them.
qt,jI is head j's query, a vector of dI=128 numbers made from token t.
ksI is the indexer key of token s, also 128 numbers, and the same for every head.
wt,jI is one number per head, also made from token t: how much this query trusts head j.
A tiny example by hand. Take 2 heads and vectors of 2 numbers. The query token makes q1=(1,0), q2=(0,1) and weights w=(2,1). There are three earlier tokens with keys kA=(3,−1), kB=(−2,2) and kC=(1,1).
Token
q1⋅k
ReLU
q2⋅k
ReLU
I=2×ReLU1+1×ReLU2
A
3
3
-1
0
6
B
-2
0
2
2
2
C
1
1
1
1
3
With k=2, the selection keeps A and C. Head 1 liked A strongly, and head 2's dislike of A was cut to 0 by ReLU, not counted against it. Each head can vote for tokens, but it cannot veto them. The per-query weights then decide whose votes count more.
The real thing. The V3.2-Exp reference code on Hugging Face shows how each piece is built. This is the core of its Indexer.forward, unchanged:
python
q = self.wq_b(qr) # qr: MLA's 1,536-number query latentq = q.view(bsz, seqlen, self.n_heads, self.head_dim) # 64 heads x 128...k = self.wk(x) # x: the 7,168-number hidden state -> ONE 128-number keyk = self.k_norm(k)...q = rotate_activation(q) # a Hadamard rotation before 8-bit roundingk = rotate_activation(k)q_fp8, q_scale = act_quant(q, block_size, self.scale_fmt)k_fp8, k_scale = act_quant(k, block_size, self.scale_fmt)self.k_cache[:bsz, start_pos:end_pos] = k_fp8 # the indexer keeps its own FP8 key cache...weights = self.weights_proj(x.float()) * self.n_heads ** -0.5...topk_indices = index_score.topk(min(self.index_topk, end_pos), dim=-1)[1]
Four details are worth seeing. First, the indexer query is not made from scratch. It reuses MLA's compressed query, so it costs one extra matrix, 1,536 by 8,192 numbers. Second, the key is one 128-number vector per token, with a normalisation, and RoPE on its first 64 numbers (left out above).
Third, both are rotated by a Hadamard transform and then rounded to FP8. Our reading: the rotation spreads large outlier values over all 128 numbers, so 8-bit rounding loses less. Fourth, the indexer keeps its own key cache in FP8, next to MLA's cache. Remember that cache; it comes back when we count bytes.
With k=2,048 and the selection in place, the attention of query token t becomes:
ut=Attn(ht,{cs∣It,s∈Top-k(It,:)})
Here ut is the attention output for token t and ht is its hidden state. The set holds the cached entries whose index score is among the k largest. For a query at position t≤2,048 the set holds every earlier token, so short contexts get dense attention.
Why can DSA pick single scattered tokens when NSA insisted on blocks? Because of the MQA mode.
One query token through DSA in one layer of DeepSeek-V3.2. The indexer path is cheap but touches every earlier token. The core attention is expensive but touches only the 2,048 selected latents.
Per (query, key) pair and layer, the indexer does 64×128=8,192 multiply-adds, in FP8. MQA-mode MLA does 139,264, in BF16. That is 17 times fewer operations, and FP8 arithmetic runs about twice as fast. So the quadratic part is roughly 30 to 35 times cheaper per pair. This is our estimate from the configuration, not a number from the paper.
Now redo the prefill count for a whole prompt, with the calculator:
plain text
PREFILL: a whole prompt of L tokens, whole model, attention core only (TFLOP) L dense MHA dense MQA DSA idx DSA main DSA tot MHA/DSA weights 8,192 167.7 570.2 33.5 249.4 283.0 0.6x 606.2 16,384 670.7 2,280.5 134.1 534.5 668.6 1.0x 1,212.4 32,768 2,682.9 9,121.8 536.6 1,104.6 1,641.2 1.6x 2,424.8 65,536 10,731.4 36,486.8 2,146.3 2,244.8 4,391.1 2.4x 4,849.7 131,072 42,925.3 145,945.9 8,585.1 4,525.2 13,110.2 3.3x 9,699.3Per-query break-even, dense MHA-mode vs DSA: t = 8,704 tokens (7,737 if the FP8 indexer counts half)Below that, a dense pass is cheaper: why V3.2 runs short prefills in a masked MHA mode.
Look at the 8K row. DSA costs more than dense prefill: 283 against 168 trillion FLOPs. The reason is the MHA/MQA split from earlier. Dense prefill uses the cheap MHA mode at 40,960 multiply-adds per pair. DSA's selected attention uses MQA mode at 139,264 per pair, 3.4 times more. At short lengths, the saving from skipping tokens is smaller than that penalty.
The break-even per query follows from one equation. Dense MHA mode costs 40,960t for a query at position t. DSA costs 8,192t+139,264×2,048. They are equal at t=8,704, or 7,737 if FP8 counts half.
Below that, dense is cheaper. That is exactly the masked MHA mode in the paper. For short prompts, run dense and mask out the unselected tokens. It gives the same answer for less work.
At 128K, DSA's prefill is 3.3 times cheaper than dense MHA mode, by our count.
Attention compute for a whole prompt of DeepSeek-V3 (our calculator). Dense MHA-mode prefill grows with the square of the length. DSA starts more expensive and wins after a few thousand tokens. At 128K it is 3.3 times cheaper.
Teaching the indexer: two stages of continued training#
A new indexer starts with random weights. It must learn which tokens the model's own attention cares about. DeepSeek teaches it by imitation, in two stages, starting from the V3.1-Terminus checkpoint with its 128K context.
The path from V3.1-Terminus to V3.2. A short dense warm-up trains only the indexer. A long sparse stage trains everything with top-2,048 selection switched on. Post-training then runs with DSA as well. The bar shows the tokens of the two stages to scale.
The first term dominates: the token the teacher cares about most is under-scored. The gradient pushes the indexer to raise that token's score. Note the direction of the KL. With p first, missing a token the teacher needs costs a lot, while giving a little score to an unneeded token costs little. For a selector that keeps 2,048 tokens, that is the right bias.
Check the arithmetic again: 15,000 steps times 480 sequences is 7.2 million sequences, and times 131,072 tokens that is 943.7 billion tokens. The warm-up is 0.2% of the total, as the bar in the figure shows. The data in both stages is "totally aligned" with V3.1-Terminus's 128K long-context extension data, so the only new thing the model sees is the sparsity.
Attention, Part 3 trained an indexer on a real 0.5B model. Here we do something smaller and more controlled. We build the whole DSA path from scratch on random tensors, with V3.2's shape in miniature, and measure everything. The script is part5_indexer.py and runs on a laptop CPU in under a minute.
The setting has 4,096 cached tokens and 16 query heads that share one 32-number latent per token (MQA mode). The indexer has 16 heads of 16 numbers. Each query keeps k=64 tokens. That is 1.56% of the cache, the same share as 2,048 of 131,072 in V3.2. The "main model" is two fixed random matrices. The hidden states are random, plus one shared mean direction, because real hidden states are not centred on zero.
The indexer is the formula, line for line:
python
class Indexer(torch.nn.Module): """I[t,s] = sum_j w[t,j] * ReLU(q_I[t,j] . k_I[s]).""" def __init__(self): super().__init__() self.wq = torch.nn.Linear(D, HI * DI, bias=False) # query token -> HI small queries self.wk = torch.nn.Linear(D, DI, bias=False) # earlier token -> ONE small key self.ww = torch.nn.Linear(D, HI, bias=False) # query token -> one weight per head def forward(self, h_q, h_kv, fp8=False): B = h_q.shape[0] q = self.wq(h_q).view(B, T, HI, DI) # (B,T,HI,DI) k = self.wk(h_kv) # (B,L,DI) if fp8: # store and multiply in 8-bit floats q = q.to(torch.float8_e4m3fn).float() k = k.to(torch.float8_e4m3fn).float() w = self.ww(h_q) # (B,T,HI) dots = torch.einsum('btjd,bsd->btjs', q, k) # (B,T,HI,L) return torch.einsum('btj,btjs->bts', w, F.relu(dots)) # (B,T,L): the index scores
The sparse attention gathers the 64 selected latents and attends over them only:
python
def sparse_attention(q, c, idx): """Method 1: gather the k selected latents and attend over them only. idx: (B,T,K).""" B = q.shape[0] sel = torch.gather(c.unsqueeze(1).expand(B, T, L, DC), 2, idx.unsqueeze(-1).expand(B, T, K, DC)) # (B,T,K,DC) s = TEMP * torch.einsum('bhtc,btkc->bhtk', q, sel) / math.sqrt(DC) a = torch.softmax(s, -1) return torch.einsum('bhtk,btkc->bthc', a, sel) # (B,T,H,DC)
Then the script runs the dense warm-up of Stage 1. The main model is frozen, and only the indexer trains, with DKL(p∥Softmax(I)), for 400 steps on fresh random sequences. It tests on a fixed batch that it never trained on. Here is the full output:
plain text
Setup: 4,096 cached tokens, 32 queries per sequence, 16 heads sharing one 32-dim latent, indexer 16 heads x 16 dims, keep k = 64 (1.56%)Dense attention: the best 64 tokens hold 90.1% of the head-summed attention on averagewarm-up step 1: KL loss 11.653 per querywarm-up step 25: KL loss 4.727 per querywarm-up step 100: KL loss 2.308 per querywarm-up step 200: KL loss 1.855 per querywarm-up step 400: KL loss 1.134 per querywarm-up took 37 s on CPUselector attention kept recall of true top-k output errorrandom tokens 1.5% 1.6% 71.9%untrained indexer 2.0% 2.0% 74.1%trained indexer 77.1% 56.6% 17.2%oracle (true top-k) 90.1% 100.0% 5.0%k = 16 ( 0.39% of tokens): attention kept, trained indexer 52.9%, oracle 73.2%, random 0.4%k = 32 ( 0.78% of tokens): attention kept, trained indexer 65.6%, oracle 83.0%, random 0.8%k = 64 ( 1.56% of tokens): attention kept, trained indexer 77.1%, oracle 90.1%, random 1.6%k = 128 ( 3.12% of tokens): attention kept, trained indexer 86.2%, oracle 94.8%, random 3.1%k = 256 ( 6.25% of tokens): attention kept, trained indexer 92.8%, oracle 97.7%, random 6.2%k = 512 (12.50% of tokens): attention kept, trained indexer 97.0%, oracle 99.1%, random 12.5%k = 1024 (25.00% of tokens): attention kept, trained indexer 99.0%, oracle 99.8%, random 25.0%FP8 indexer (float8_e4m3fn q and k): 95.2% of the selected tokens are the same as in FP32Method 1 (gather 64 latents) vs Method 2 (SDPA with a mask over all 4,096): max difference 4.8e-06Multiply-adds per query token: dense attention: 16 x (32+32) x 4,096 = 4,194,304 DSA indexer: 16 x 16 x 4,096 = 1,048,576 DSA main (top-64): 16 x (32+32) x 64 = 65,536 DSA total: 1,114,112 (3.8x fewer than dense)
"Attention kept" is the share of the true head-summed attention that falls on the selected tokens. "Output error" is the gap between the sparse and the dense output, relative to the dense output.
Random tokens keep 1.5% of the attention, as expected from picking 1.56% of tokens blindly. The output is badly wrong (72% error).
The untrained indexer is no better than random.
The trained indexer keeps 77.1% of the attention after 37 seconds of warm-up. The output error falls to 17.2%. It finds only 56.6% of the exact top 64. But the tokens it misses have small weights, so most of the attention mass is still covered.
The oracle, the true top 64, keeps 90.1% with a 5.0% error. That is the ceiling for any selector at this k.
Share of the true attention held by the selected tokens, against k, in our toy (4,096 cached tokens). The trained indexer closes most of the gap to the oracle as k grows. Random selection keeps only k divided by 4,096.
Three more lines answer three more questions. FP8: rounding the indexer's queries and keys to 8 bits changes only 4.8% of the selected tokens. A ranking survives coarse numbers well, which is why DeepSeek can run the indexer in FP8. Method 2: PyTorch's own scaled_dot_product_attention, given a mask that allows only the selected tokens, gives the same output as our gather to within 4.8×10−6. So the from-scratch sparse attention is correct. And the multiply-adds: 3.8 times fewer than dense here.
Why only 3.8 times, against 13.4 for V3.2 at 128K? Our toy indexer is large next to its main attention: a ratio of 4 per pair, against 17 in V3.2. And its context is 32 times shorter.
The calculator counts operations. DeepSeek measured money.
The figure has no table of numbers, but it is a vector drawing. The PDF stores each line as exact coordinates. So instead of reading the plot by eye, we read the coordinates with part5_fig3_read.py. It finds the grid lines for the scale, finds the blue and orange paths, and converts their corner points.
plain text
prefill V3.1-Terminus corner points (token position, $ per 1M tokens): [(0, 0.0529), (131072, 0.6657)]prefill V3.2 corner points (token position, $ per 1M tokens): [(0, 0.0611), (6746, 0.0999), (131072, 0.1871)]prefill at 128K: Terminus $0.666, V3.2 $0.187, ratio 3.6x; curves cross near 10,688 tokensprefill at 32K: Terminus $0.206, V3.2 $0.118; at 64K: Terminus $0.359, V3.2 $0.141decode V3.1-Terminus corner points (token position, $ per 1M tokens): [(2, 0.0706), (131070, 2.1543)]decode V3.2 corner points (token position, $ per 1M tokens): [(2, 0.0953), (2107, 0.1458), (104046, 0.2275), (131070, 0.2492)]decode at 128K: Terminus $2.154, V3.2 $0.249, ratio 8.6x; curves cross near 4,928 tokensdecode at 32K: Terminus $0.592, V3.2 $0.170; at 64K: Terminus $1.112, V3.2 $0.197
The corner points tell a story. Each curve is made of only two to four straight segments. So these are fitted cost models drawn as lines, not raw measurements with noise. Inside them, three facts stand out.
At 128K, V3.2 is 3.6 times cheaper in prefill and 8.6 times cheaper in decode. Decode gains more because decode is memory-bound, and DSA cuts reads hardest.
The decode curve of V3.2 bends at about 2,100 tokens. That is k=2,048. Below it, every token is selected anyway, and the cost rises like dense. Above it, the main attention stops growing.
The prefill curve of V3.2 bends at about 6,700 tokens, and the two prefill curves cross near 10,700. Below the bend, DeepSeek runs the masked MHA mode, which costs about the same as Terminus. Our break-even estimate from operation counts was 7,700 to 8,700 tokens, in the same range. The real crossing depends on kernels, which our count ignores.
The decode curve of V3.2 also has a small bend at about 104,000 tokens, where it gets slightly steeper. The paper does not explain it, and we do not guess.
How many times cheaper a token is on V3.2 than on V3.1-Terminus, by its position, read from the vector data of the paper's Figure 3. Prefill crosses 1x near 10,700 tokens; decode gains much more at long positions.
DeepSeek's release note for V3.2-Exp says it plainly: "DeepSeek API prices drop 50%+, effective immediately". DeepSeek also kept V3.1-Terminus on a temporary endpoint until 15 October 2025, so users could compare the two. The archived pricing pages give the exact prices in USD per million tokens:
V3.1 (from 5 Sep 2025)
V3.2-Exp (from 29 Sep 2025)
Cut
Input, cache hit
$0.07
$0.028
60%
Input, cache miss
$0.56
$0.28
50%
Output
$1.68
$0.42
75%
DeepSeek API prices per million tokens for V3.1 and for V3.2-Exp, from the archived pricing pages before and after the launch on 29 September 2025.
The output price fell the most, and decode is where Figure 3 shows the biggest saving. That fits, but keep the two things apart. A price is a business decision; Figure 3 is DeepSeek's own cost estimate at an assumed rental price.
The price is flat across positions, while the cost saving depends on context length. Short chats save little; long agent sessions and long reasoning chains save a lot. DeepSeek kept the same prices for V3.2 and V3.2-Speciale in December 2025.
It is easy to read "sparse attention" as "smaller cache". For DSA, that is wrong. Every token's MLA latent must stay in memory, because any of them might be selected by a later query. And the indexer adds its own key per token per layer.
plain text
KV cache per token (61 layers): V3, BF16 latent: 61 x 576 x 2 B = 70,272 B V3, FlashMLA FP8 row: 61 x 656 B = 40,016 B V3.2, FP8 row + indexer: 61 x (656 + 132) B = 48,068 B (+20.1%) at 128K tokens: V3 FP8 5.24 GB, V3.2 6.30 GB per sequence
The indexer key is 128 FP8 bytes plus one 4-byte scale per layer: 132 bytes. Added to the 656-byte FP8 row of MLA, V3.2 stores 48,068 bytes per token. This number is not only ours. The DeepSeek-V4.1-Flash paper's Figure 1 shows a bar for "DeepSeek-V3.2" at exactly 48,068 bytes per token. Our sum from the code matches it to the byte.
Bytes of cache per token. V3's BF16 latent, the same latent in DeepSeek's FP8 serving layout, and V3.2, which adds a 132-byte indexer key per layer. DSA makes the cache 20% larger, not smaller.
So what does DSA save? Look again at the bytes table from the start. At 128K, a new token reads 1,055 MB of indexer keys and only 82 MB of selected latents. That is 1,137 MB in all, against 5,245 MB for dense. Reads fall 4.6 times.
But 93% of what DSA still reads is the indexer's key cache, and that part still grows with the context. Decode with DSA is still linear in L; the slope is about 5 times smaller.
That remaining linear term, the full-length indexer pass, is exactly what DeepSeek attacked next. V4's compressed sparse attention shrinks the cache itself, and V4.1 goes further. Part 6 follows that path.
On 1 December 2025 DeepSeek released V3.2 and V3.2-Speciale, both under the MIT license, and the arXiv paper followed the next day. The architecture is the same as V3.2-Exp: DSA on MLA, 671B parameters, 37B active, 128K context. The difference is post-training. The paper says post-training also "employs sparse attention in the same way as the sparse continued pre-training stage".
Specialist models for maths, coding, reasoning and agent tasks generate training data. Then one mixed reinforcement-learning stage with GRPO trains reasoning, agent skills and alignment together. DeepSeek says its RL budget now "exceeds 10% of the pre-training cost". Part 4 covers that training in detail.
The paper reports Speciale at gold-medal level in four 2025 competitions. It scored 35 of 42 points at the IMO and 492 of 600 at the IOI. It also reached gold at the Chinese Mathematical Olympiad and solved 10 of 12 problems at the ICPC World Finals. It scored 96.0 on AIME 2025.
Speciale had no tool calls. It was served only through a temporary API endpoint until 15 December 2025, at the same price as V3.2. The regular V3.2 became the model behind DeepSeek's deepseek-chat and deepseek-reasoner API names.
Research notes: sources, versions and loose ends
Which document. V3.2-Exp came with its own technical report as a PDF in the GitHub repository (29 September 2025), "DeepSeek-V3.2-Exp: Boosting Long-Context Efficiency with DeepSeek Sparse Attention". The arXiv paper 2512.02556 (2 December 2025) repeats its DSA sections and extends them. All quotes and page numbers here are from the arXiv version.
NSA training tokens. The NSA paper says the 27B model was pre-trained on "260B tokens" in §1 (page 3) and on "270B tokens of 8k-length texts" in §4.1 (page 10). We do not know which is right.
NSA decode speed. The 11.6x for decoding is the memory-access ratio of Table 4 ("expected speedup"), not a timed kernel. The forward (9.0x) and backward (6.0x) numbers are timed, on A100 GPUs with Triton kernels.
Indexer size. From the V3.2-Exp model.py shapes: wq_b 1,536 x 8,192, wk 7,168 x 128, weights_proj 7,168 x 64, plus a LayerNorm of 128. About 14.0M weights per layer, 0.85B over 61 layers. The model card still calls V3.2 a 671B model.
The 48,068 bytes. The indexer key cache layout (128 FP8 bytes + one float32 scale per 128 numbers) comes from the k_cache and k_scale_cache buffers in model.py. The MLA row (512 FP8 bytes, 16 bytes of scales, 128 bytes of BF16 RoPE key) comes from FlashMLA's README at commit ba89a34. The sum matches the V4.1-Flash paper's Figure 1(b).
Figure 3 values. Read from the PDF's vector paths, not by eye. The strokes of the decode panel stop 2 tokens short of the axis ends; we clamp to the drawn range.
Our calculator's limits. It counts the attention core only, in multiply-adds. It ignores the indexer's projections, softmax, top-k sorting, and kernel efficiency. It treats FP8 and BF16 operations as equal unless stated. "Weights" uses 2 x 37B FLOPs per token.
The API cut. Verified on DeepSeek's release note (news250929): "API prices cut by 50%+". The exact prices come from Wayback Machine captures of the pricing page on 6 September 2025 and 4 October 2025.
Next:Part 6, V4 and V4.1. DSA left the cache as large as before and the indexer still reading every token. V4 compresses the cache itself with hybrid compressed and sparse attention, and reaches a million tokens of context.
Our code, in code/deepseek/: part5_cost.py (the calculator), part5_indexer.py (the from-scratch indexer), part5_fig3_read.py (Figure 3 from the PDF's vectors).