Ishwar Jangid

systems, machine learning, and GPUs. learning in public.

I am a software engineer who builds production AI systems: agents, retrieval pipelines and the data platforms underneath them.

I learn by writing things down properly. This site is where those notes become long-form, first-principles books on GPU programming, training LLMs from scratch, retrieval-augmented generation and ML systems, each with code you can run and numbers you can measure.

Writings

• Serving LLMs in Production: Latency, Capacity, Cost, and What Breaks

  TTFT, goodput and honest benchmarks; queueing maths and capacity planning for an 8B model on an H100; cost per million tokens; quantization measured on a real model; routing, multi-LoRA, autoscaling and the failures to watch for.

• Beyond One GPU: Tensor, Pipeline and Expert Parallelism, and Split Prefill and Decode

  Why big models need many GPUs, what the wires between them cost, and how tensor, pipeline and expert parallelism and disaggregated serving split the work: a real two-process tensor-parallel run, measured MoE routing, and worked arithmetic.

• SGLang Explained: How It Reuses Work, and How It Differs from vLLM

  RadixAttention from a handbook example: KV reuse maths, real prefill timings, cache and scheduling simulations, and a fair comparison with today's vLLM.

• Attention, Part 1: Self-Attention and Multi-Head Attention

  The one idea every modern AI language model is built on, explained from zero: why attention was invented, how a word decides which other words to look at, why the scores are divided by √d, how several heads work together, and what real attention heads inside Qwen2.5 actually learn. Every claim checked in code.

• Attention, Part 2: MQA, GQA and MLA

  Every token leaves its keys and values behind in memory (the KV cache), and plain multi-head attention leaves a lot. How multi-query, grouped-query and multi-head latent attention shrink it, explained from zero, built from scratch, checked against a real model's layer, and measured: DeepSeek-V3 stores 68.6 KiB per token where plain attention would need 4,880.

• Attention, Part 3: Sliding-Window and Sparse Attention

  Why looking at every earlier token gets too expensive, and the three ways today's models avoid it: sliding windows (Mistral), mixing local and global layers (Gemma 2, Gemma 3, gpt-oss), and letting the model pick its own tokens (DeepSeek Sparse Attention). Explained from zero, with equations, quotes from the papers, and real experiments: a lightning indexer trained on Qwen2.5-0.5B keeps quality close to full attention while each token reads only 64 of up to 2,048 earlier tokens.

• Attention, Part 4: Linear Attention, Gated DeltaNet and Hybrid Models

  The newest idea in attention: replace the ever-growing KV cache with a fixed-size memory. Linear attention, the delta rule, forget gates, Gated DeltaNet, Kimi Delta Attention, gated attention, and the hybrid models built from them (Qwen3-Next, Kimi Linear, Nemotron-H). Explained from zero with equations, paper screenshots and real runs, including three small models trained side by side.

• How an LLM Writes: Prefill and Decode

  Why the first word takes a moment and the rest stream steadily. The two phases of LLM inference, built from the ground up and measured on a real model: a 512-token prompt is read in 27 ms, but writing 512 tokens takes 5.2 seconds.

• The KV Cache: An LLM's Short-Term Memory, and Its Bill

  Why every LLM keeps notes on each token it has read, how big those notes get (16 GiB for one 128K-token Llama 3.1 8B conversation), how models shrink them, and why they, not compute, usually decide how many people one GPU can serve.

• How vLLM Serves Thousands: Paging, Batching and Caching

  PagedAttention, continuous batching, chunked prefill and prefix caching, each built from the problem it solves and measured or simulated with real costs: from 48 to 415 requests on one GPU, pauses cut from 463 ms to 40 ms, and a 7x faster first token.

• Speculative Decoding: Several Tokens per Step, Not One Word Changed

  How a small model can guess ahead and a big model can check all the guesses at once, why the output stays exactly the same, and what it really buys on real hardware: speculative decoding built from scratch and measured, from 1.16x on prose to 1.89x on code and 3.49x when the answer copies the prompt.

• HNSW, From the Ground Up

  How Hierarchical Navigable Small World graphs find nearest neighbours in microseconds: the two ideas they combine, how the graph is built, what M, efConstruction and efSearch really cost, and measured numbers from a 200,000-vector index.

• How this site renders Markdown

  A tour of every element the reader supports (headings, code, callouts, tables, math and diagrams), so writing a new chapter is just dropping a .md file into content/.

Videos

• Jev: The AI Model That Doesn't Talk. It Decides.

  What Jev, TypeSafe's decision model, actually is, where it fits next to plain code, a classifier and an LLM, and three real uses: an agent router that makes four decisions in one call, a cheap gate before a big model, and a guardrail around an LLM. Every number from real API runs.

• How Vector Search Really Works: HNSW, Filters and Qdrant

  Vectors in Depth, Part 1. How a vector database finds the nearest meaning among millions of points in a fraction of a millisecond: embeddings, the HNSW graph and its three knobs, and the hard part, making approximate geometry obey exact filters. Every number from real Qdrant runs.

Research papers

• Circuits, explained

  The 2021 paper that started reverse-engineering transformers by hand. It writes a small attention-only model as a sum of simple pieces you can read straight from the weights: bigram tables, skip-trigrams and induction heads. Explained in plain English, with every equation derived, the paper's own figures, and real code on small open models.

• ViT, explained

  The 2020 paper that cut an image into 16×16 patches, fed them to an unchanged Transformer, and beat the best convolutional networks once it was given enough data. Explained line by line in six parts, with the paper's own text, simple pictures, worked equations and real code on the released models.

• BERT, explained

  The 2018 paper that taught one model to read text in both directions at once, then reuse that skill for almost any language task. Explained line by line in six parts, with the paper's own text, simple pictures and real code.

Books

• GPU Programming

  A complete course that teaches you to predict how fast a kernel can go, measure how fast it does go, and close the gap. From CPU parallelism to Hopper, Blackwell, FlashAttention and multi-GPU inference.

• RAG: From First Principles

  Retrieval-augmented generation built and measured step by step. Embeddings, chunking, hybrid search, reranking, evaluation, hallucination, agentic RAG with LangGraph, and ingesting a million PDFs.

• Agentic Systems

  How real AI agents are designed, measured and run in production: the agent loop, tools and memory, the reasoning and workflow patterns from the research papers, multi-agent systems, evals that actually catch regressions, tracing, guardrails, and what companies like Uber, DoorDash and Anthropic learned the hard way. Written for engineers who know what an LLM call is and want to build systems around it.

• LLM from Scratch

  Build a language model end to end (tokenizer, data pipeline, architecture, pretraining, fine-tuning, mixture of experts, reasoning and RL with GRPO), then learn how frontier labs do the same thing at scale.

• How Models Are Trained

  How a language model goes from predicting the next word to following instructions, holding a conversation and reasoning step by step. The history from 2017 to today, the research papers as highlighted screenshots, every equation explained symbol by symbol, and real code run on real models, in plain English for beginners.

• LangGraph & LangChain

  A doc-verified, code-first book on LangGraph 1.2 and LangChain 1.4: models, messages, graphs, checkpointers, middleware, multi-agent systems and a production architecture review.

Projects

• Manim Studio

  An explainer-video pipeline: animation, voice and code generation in one place