Writings
- Every DeepSeek Model, and What Is Really Open
Every DeepSeek model from 2023 to 2026 with its size, context and license, and exactly what you can download, run or rebuild yourself.
- The DeepSeek-V3 Architecture, Piece by Piece
How DeepSeek-V3 holds 671 billion parameters but uses 37 billion per token: many small experts, a fair router, a compressed memory and a second prediction head.
- How DeepSeek-V3 Was Trained on 2,048 GPUs
How DeepSeek-V3 was trained: 14.8T tokens on 2,048 GPUs, a schedule that hides network time, 8-bit numbers, and what the $5.576M covers.
- DeepSeek-R1 and Reasoning by Reinforcement Learning
How DeepSeek taught models to reason with rewards a program can check: GRPO by hand, R1-Zero, the four-stage R1 recipe, distillation, costs, and what came after.
- Sparse Attention in DeepSeek: NSA, DSA and V3.2
How DeepSeek stopped each new token from reading the whole conversation: NSA, the token scorer of V3.2, its measured costs and the price cut.
- DeepSeek-V4 and V4.1: A Million Tokens of Context
How DeepSeek cut the memory per token of context from 389,120 bytes to 890: compressed attention, stable wide residuals, FP4 weights and hashed n-gram memory.
- Running DeepSeek on Your Own Hardware, and How DeepSeek Serves It
How much memory each DeepSeek model needs, which GPUs, Macs and CPU servers run it at what reported speed, and how DeepSeek serves it at scale.
- DeepSeek's Specialised Models: Code, Maths, Proofs, Vision and OCR
The side models behind DeepSeek's main line: Coder, DeepSeekMath and GRPO, the Lean provers, VL2, Janus and DeepSeek-OCR, built up from zero with small experiments.
- vLLM on a Free GPU: Run It, Load It, Break It
A real vLLM server on a free Colab T4, measured like a production service: KV cache, batching, the latency knee, prefix caching, preemption, failures.
- DeepInfer: How DeepSeek Serves Its Models, and How It Differs from vLLM and SGLang
A code-level tour of DeepSeek's open-sourced production inference engine: its scheduler, 890-byte KV blocks, expert balancing and DSpark, next to vLLM and SGLang.
- How Ray-Ban Display Glasses Put a Screen in a Lens
How a picture appears inside a clear glasses lens: light, a tiny projector, mirrors hidden in glass, a wristband that reads muscles, and the whole system.
- Reinforcement Learning: From Trial and Error to AlphaZero
How machines learned from reward alone: cats in puzzle boxes, Bellman's equation, backgammon, Atari, AlphaGo and the methods later used to train chat models.
- From RLHF to Reasoning: How Language Models Are Trained Today
How chat and reasoning models are taught after pre-training: human preference labels, AI feedback, answers a program can check, and how we test the result.
- Inside a Modern Language Model: Data, Scale and Architecture
How a frontier language model is built before any chat training: the web data, the scaling rules, the layer design, experts, and the machines that train it.
- From One Neuron to Opus: How AI Got Here
Eighty years of AI in one story: the first artificial neuron, the AI winters, deep learning, the Transformer, and the road to today's Claude models.
- Classical Machine Learning: From Least Squares to XGBoost
The other branch of AI's history: regression, nearest neighbours, Naive Bayes, decision trees, SVMs, random forests and boosting, and why trees still win on tables.
- Serving LLMs in Production: Latency, Capacity, Cost, and What Breaks
TTFT, goodput and honest benchmarks; queueing maths and capacity planning for an 8B model on an H100; cost per million tokens; quantization measured on a real model; routing, multi-LoRA, autoscaling and the failures to watch for.
- Beyond One GPU: Tensor, Pipeline and Expert Parallelism, and Split Prefill and Decode
Why big models need many GPUs, what the wires between them cost, and how tensor, pipeline and expert parallelism and disaggregated serving split the work: a real two-process tensor-parallel run, measured MoE routing, and worked arithmetic.
- SGLang Explained: How It Reuses Work, and How It Differs from vLLM
RadixAttention from a handbook example: KV reuse maths, real prefill timings, cache and scheduling simulations, and a fair comparison with today's vLLM.
- Attention, Part 1: Self-Attention and Multi-Head Attention
The one idea every modern AI language model is built on, explained from zero: why attention was invented, how a word decides which other words to look at, why the scores are divided by √d, how several heads work together, and what real attention heads inside Qwen2.5 actually learn. Every claim checked in code.
- Attention, Part 2: MQA, GQA and MLA
Every token leaves its keys and values behind in memory (the KV cache), and plain multi-head attention leaves a lot. How multi-query, grouped-query and multi-head latent attention shrink it, explained from zero, built from scratch, checked against a real model's layer, and measured: DeepSeek-V3 stores 68.6 KiB per token where plain attention would need 4,880.
- Attention, Part 3: Sliding-Window and Sparse Attention
Why looking at every earlier token gets too expensive, and the three ways today's models avoid it: sliding windows (Mistral), mixing local and global layers (Gemma 2, Gemma 3, gpt-oss), and letting the model pick its own tokens (DeepSeek Sparse Attention). Explained from zero, with equations, quotes from the papers, and real experiments: a lightning indexer trained on Qwen2.5-0.5B keeps quality close to full attention while each token reads only 64 of up to 2,048 earlier tokens.
- Attention, Part 4: Linear Attention, Gated DeltaNet and Hybrid Models
The newest idea in attention: replace the ever-growing KV cache with a fixed-size memory. Linear attention, the delta rule, forget gates, Gated DeltaNet, Kimi Delta Attention, gated attention, and the hybrid models built from them (Qwen3-Next, Kimi Linear, Nemotron-H). Explained from zero with equations, paper screenshots and real runs, including three small models trained side by side.
- How an LLM Writes: Prefill and Decode
Why the first word takes a moment and the rest stream steadily. Prefill, decode, the GPU's two speeds, the roofline and batching, measured on a real model: reading 512 tokens takes 27 ms, writing 512 takes 5.2 s.
- The KV Cache: An LLM's Short-Term Memory, and Its Bill
Why an LLM keeps notes on every token, how big they get (16 GiB for one 128K-token Llama 3.1 8B chat), and why the old way of storing them wasted most of the memory.
- How vLLM Serves Thousands: Paging, Batching and Caching
How vLLM stores the KV cache in small pages, shares and reuses them, and schedules every step: from 48 to 415 requests on one GPU, pauses cut from 463 ms to 40 ms, a 7x faster first token.
- Speculative Decoding: Several Tokens per Step, Not One Word Changed
How a small model can guess ahead and a big model can check all the guesses at once, why the output stays exactly the same, and what it really buys on real hardware: speculative decoding built from scratch and measured, from 1.16x on prose to 1.89x on code and 3.49x when the answer copies the prompt.
- HNSW, From the Ground Up
How Hierarchical Navigable Small World graphs find nearest neighbours in microseconds: the two ideas they combine, how the graph is built, what M, efConstruction and efSearch really cost, and measured numbers from a 200,000-vector index.