vLLM, read beside the paper

The paper and its explanation side by side.

  • Part 1: The Big Idea: Memory, Not Arithmetic, Limits LLM Serving

    How a chat server answers many users at once, why its GPU runs out of memory before it runs out of speed, and the paper's fix: store each request's memory in small pages. We read the title, the abstract, Figures 1 and 2 and §1, and we measure the KV cache of a real model.

  • Part 2: Why Memory Is the Bottleneck

    How a language model writes text, why one GPU must serve many requests at once, and where the memory for those requests goes. We read §2, §3 and Figure 3, with runs on a real model.

  • Part 3: PagedAttention

    How vLLM stores each request's keys and values in small blocks that can sit anywhere in memory, and still computes the same attention. We read §4.1 to §4.3 and build a block table that drives a real model.

  • Part 4: Sharing, Scheduling and Many GPUs

    How vLLM lets several answers share one prompt's memory, what it does when memory runs out, and how it runs one model on many GPUs. We read §4.4 to §4.6 and §5, and build each piece in Python.

  • Part 5: The Experiments

    How the paper tests vLLM against FasterTransformer and three versions of Orca, how to read its latency plots, and what the ablations show. We rebuild the main experiments as a small serving simulation on real request lengths.

  • Part 6: Build It from Scratch

    We write a small vLLM in about 500 lines of Python: blocks, block tables, PagedAttention, a scheduler and parallel sampling. It runs the real OPT-125M, matches the transformers library and serves real requests.