vLLM, read beside the paper
The paper and its explanation side by side.
- Part 1: The Big Idea: Memory, Not Arithmetic, Limits LLM Serving
How a chat server answers many users at once, why its GPU runs out of memory before it runs out of speed, and the paper's fix: store each request's memory in small pages. We read the title, the abstract, Figures 1 and 2 and §1, and we measure the KV cache of a real model.
- Part 2: Why Memory Is the Bottleneck
How a language model writes text, why one GPU must serve many requests at once, and where the memory for those requests goes. We read §2, §3 and Figure 3, with runs on a real model.
- Part 3: PagedAttention
How vLLM stores each request's keys and values in small blocks that can sit anywhere in memory, and still computes the same attention. We read §4.1 to §4.3 and build a block table that drives a real model.
- Part 4: Sharing, Scheduling and Many GPUs
How vLLM lets several answers share one prompt's memory, what it does when memory runs out, and how it runs one model on many GPUs. We read §4.4 to §4.6 and §5, and build each piece in Python.
- Part 5: The Experiments
How the paper tests vLLM against FasterTransformer and three versions of Orca, how to read its latency plots, and what the ablations show. We rebuild the main experiments as a small serving simulation on real request lengths.
- Part 6: Build It from Scratch
We write a small vLLM in about 500 lines of Python: blocks, block tables, PagedAttention, a scheduler and parallel sampling. It runs the real OPT-125M, matches the transformers library and serves real requests.