https://ishwarj.com/Ishwar Jangid writes long-form, first-principles guides and books on LLM inference, retrieval-augmented generation, GPU programming and ML systems, with code you can run and numbers you can measure.en https://ishwarj.com/writings/llm-inference-5-sglang-vs-vllm/https://ishwarj.com/writings/llm-inference-5-sglang-vs-vllm/Thu, 08 Oct 2026 00:00:00 GMTFollow a handbook assistant from its first request to a shared KV cache. Learn RadixAttention step by step, then compare SGLang with vLLM. https://ishwarj.com/papers/vit/part-1-the-big-idea/https://ishwarj.com/papers/vit/part-1-the-big-idea/Tue, 06 Oct 2026 00:00:00 GMTThe title, the abstract and the introduction of the Vision Transformer paper, line by line: what it means to cut a picture into 16×16 patches and read them like words, why convolutional networks ruled computer vision, what inductive bias, locality and translation equivariance are, and the claim that enough data beats built-in assumptions. With a real run of ViT-B/16 and ResNet-50 on the same picture and the quadratic cost of attention worked out by hand. https://ishwarj.com/papers/vit/part-2-the-model/https://ishwarj.com/papers/vit/part-2-the-model/Tue, 06 Oct 2026 00:00:00 GMTSection 2, Figure 1, Section 3.1 and Appendix A, line by line: the earlier attempts to put attention on images, then every step from a 224×224 picture to a class label, with Equations 1 to 8 recomputed by hand on the released ViT-B/16 and matched against the library at every stage, the 86,567,656 parameters counted exactly, and a four-token attention example small enough to check with a pencil. https://ishwarj.com/writings/attention-1-self-attention/https://ishwarj.com/writings/attention-1-self-attention/Sun, 04 Oct 2026 00:00:00 GMTThe one idea every modern AI language model is built on, explained from zero: why attention was invented, how a word decides which other words to look at, why the scores are divided by √d, how several heads work together, and what real attention heads inside Qwen2.5 actually learn. Every claim checked in code. https://ishwarj.com/writings/attention-2-mqa-gqa-mla/https://ishwarj.com/writings/attention-2-mqa-gqa-mla/Sun, 04 Oct 2026 00:00:00 GMTEvery token leaves its keys and values behind in memory (the KV cache), and plain multi-head attention leaves a lot. How multi-query, grouped-query and multi-head latent attention shrink it, explained from zero, built from scratch, checked against a real model's layer, and measured: DeepSeek-V3 stores 68.6 KiB per token where plain attention would need 4,880. https://ishwarj.com/writings/attention-3-sliding-window-sparse/https://ishwarj.com/writings/attention-3-sliding-window-sparse/Sun, 04 Oct 2026 00:00:00 GMTWhy looking at every earlier token gets too expensive, and the three ways today's models avoid it: sliding windows (Mistral), mixing local and global layers (Gemma 2, Gemma 3, gpt-oss), and letting the model pick its own tokens (DeepSeek Sparse Attention). Explained from zero, with equations, quotes from the papers, and real experiments: a lightning indexer trained on Qwen2.5-0.5B keeps quality close to full attention while each token reads only 64 of up to 2,048 earlier tokens. https://ishwarj.com/writings/attention-4-linear-hybrid/https://ishwarj.com/writings/attention-4-linear-hybrid/Sun, 04 Oct 2026 00:00:00 GMTThe newest idea in attention: replace the ever-growing KV cache with a fixed-size memory. Linear attention, the delta rule, forget gates, Gated DeltaNet, Kimi Delta Attention, gated attention, and the hybrid models built from them (Qwen3-Next, Kimi Linear, Nemotron-H). Explained from zero with equations, paper screenshots and real runs, including three small models trained side by side. https://ishwarj.com/papers/bert/part-1-the-big-idea/https://ishwarj.com/papers/bert/part-1-the-big-idea/Sun, 04 Oct 2026 00:00:00 GMTThe title, the abstract and the introduction of the BERT paper, line by line: what pre-training is, the two ways to reuse a pre-trained model, why reading only left to right is a real limit, and BERT's fix of filling in blanks. With a real run of GPT-2 and BERT on the same sentences. https://ishwarj.com/papers/bert/part-2-architecture-and-input/https://ishwarj.com/papers/bert/part-2-architecture-and-input/Sun, 04 Oct 2026 00:00:00 GMTSection 2 and the model half of Section 3, line by line: the ideas BERT builds on, the stack of Transformer layers, where the 110 million parameters come from (counted in the real model), WordPiece tokens, [CLS], [SEP], and the three embeddings that are added together for every token. https://ishwarj.com/papers/bert/part-3-pre-training/https://ishwarj.com/papers/bert/part-3-pre-training/Sun, 04 Oct 2026 00:00:00 GMTSection 3.1 and Appendices A.1 and A.2 of the BERT paper, line by line: why a normal language model cannot look both ways, the masked language model and its 80/10/10 rule, next sentence prediction, the training data and the full training recipe. Every claim run on the real model or measured in code. https://ishwarj.com/papers/bert/part-4-fine-tuning-and-results/https://ishwarj.com/papers/bert/part-4-fine-tuning-and-results/Sun, 04 Oct 2026 00:00:00 GMTSections 3.2 and 4 of the BERT paper, line by line: how one pre-trained model becomes a classifier, a question answerer and a multiple-choice solver by adding a tiny output layer, and what the results tables really say. With a real fine-tune of BERT-base on SST-2 and MRPC, and our own span search scored on the full SQuAD dev sets. https://ishwarj.com/papers/bert/part-5-ablations/https://ishwarj.com/papers/bert/part-5-ablations/Sun, 04 Oct 2026 00:00:00 GMTSection 5 and Appendix C of the BERT paper, line by line: which parts of BERT actually cause its gains. Next sentence prediction, bidirectionality, model size, frozen features versus fine-tuning, training length and the 80/10/10 masking recipe, every table row explained, with real parameter counts, a real perplexity and a real re-run of the feature-based NER experiment. https://ishwarj.com/papers/bert/part-6-impact-and-summary/https://ishwarj.com/papers/bert/part-6-impact-and-summary/Sun, 04 Oct 2026 00:00:00 GMTThe paper's conclusion, what came next (RoBERTa, ALBERT, DistilBERT, ELECTRA, Sentence-BERT and more), BERT's real limits measured on real models, how BERT-style models are used today, and the whole paper summarised on one page. https://ishwarj.com/writings/llm-inference-1-prefill-and-decode/https://ishwarj.com/writings/llm-inference-1-prefill-and-decode/Mon, 28 Sep 2026 00:00:00 GMTWhy the first word takes a moment and the rest stream steadily. The two phases of LLM inference, built from the ground up and measured on a real model: a 512-token prompt is read in 27 ms, but writing 512 tokens takes 5.2 seconds. https://ishwarj.com/writings/llm-inference-2-kv-cache/https://ishwarj.com/writings/llm-inference-2-kv-cache/Mon, 28 Sep 2026 00:00:00 GMTWhy every LLM keeps notes on each token it has read, how big those notes get (16 GiB for one 128K-token Llama 3.1 8B conversation), how models shrink them, and why they, not compute, usually decide how many people one GPU can serve. https://ishwarj.com/writings/llm-inference-3-vllm/https://ishwarj.com/writings/llm-inference-3-vllm/Mon, 28 Sep 2026 00:00:00 GMTPagedAttention, continuous batching, chunked prefill and prefix caching, each built from the problem it solves and measured or simulated with real costs: from 48 to 415 requests on one GPU, pauses cut from 463 ms to 40 ms, and a 7x faster first token. https://ishwarj.com/writings/llm-inference-4-speculative-decoding/https://ishwarj.com/writings/llm-inference-4-speculative-decoding/Mon, 28 Sep 2026 00:00:00 GMTHow a small model can guess ahead and a big model can check all the guesses at once, why the output stays exactly the same, and what it really buys on real hardware: speculative decoding built from scratch and measured, from 1.16x on prose to 1.89x on code and 3.49x when the answer copies the prompt. https://ishwarj.com/writings/hnsw-from-the-ground-up/https://ishwarj.com/writings/hnsw-from-the-ground-up/Sun, 27 Sep 2026 00:00:00 GMTHow Hierarchical Navigable Small World graphs find nearest neighbours in microseconds: the two ideas they combine, how the graph is built, what M, efConstruction and efSearch really cost, and measured numbers from a 200,000-vector index. https://ishwarj.com/writings/how-this-site-renders-markdown/https://ishwarj.com/writings/how-this-site-renders-markdown/Sun, 27 Sep 2026 00:00:00 GMTA tour of every element the reader supports (headings, code, callouts, tables, math and diagrams), so writing a new chapter is just dropping a .md file into content/.