# An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Author: Ishwar Jangid
Canonical URL: https://ishwarj.com/papers/vit/
Updated: 2026-10-17
Source: https://github.com/ishwar6/ishwar-books/blob/main/content/papers/vit/index.md


## Why this paper

For ten years, computer vision belonged to convolutional neural networks. Every image model, from AlexNet in 2012 to the big ResNets of 2020, was built around the convolution, a small filter slid across the picture. Meanwhile, language models had switched to the Transformer, a design with no convolutions at all, and were growing faster than anyone expected.

This paper asked a blunt question: what happens if you take the Transformer from language, change almost nothing, and feed it an image cut into 16×16 pixel squares, as if each square were a word? The answer surprised the field. On ordinary amounts of data the Transformer lost to ResNets. On very large amounts of data (14 million to 300 million images) it won, and did so with less training compute. The paper's own summary is five words: "large scale training trumps inductive bias".

The Vision Transformer (ViT) became the base of most of today's image models and of the image side of multimodal models. The paper is also unusually well written and short (nine pages plus an appendix), which makes it a good paper to read in full.

## How the six parts are organised

Each part follows the paper's own sections:

1. **[The Big Idea: An Image Is Worth 16×16 Words](https://ishwarj.com/papers/vit/part-1-the-big-idea/)**: the title, the Abstract and §1. Why convolutions dominated, what a patch is, and the claim that data beats built-in assumptions.
2. **[Inside ViT: Patches, Embeddings and the Encoder](https://ishwarj.com/papers/vit/part-2-the-model/)**: §2, §3.1, Figure 1, Equations 1 to 4 and Appendix A. The related work, and every step from pixels to a class prediction, with the real model's numbers.
3. **[Fine-tuning, Higher Resolution and the Setup](https://ishwarj.com/papers/vit/part-3-fine-tuning-and-setup/)**: §3.2, §4.1, Tables 1, 3 and 4, Appendix B.1. How a pre-trained ViT is adapted, the datasets, the baselines, and the training recipe.
4. **[Results: Beating Big CNNs, and How Much Data It Takes](https://ishwarj.com/papers/vit/part-4-results/)**: §4.2, §4.3, Table 2, Figures 2 to 4, Table 5, Appendix D.1, D.9 and D.10. Every result table and what it proves.
5. **Scaling, Looking Inside ViT, and Self-supervision** (coming soon): §4.4 to §4.6, Figures 5 to 7, Table 6, Appendix B.1.2 and D.2 to D.8. Compute versus accuracy, what the model learns, and masked patch prediction.
6. **After ViT: Impact, Limits and Summary** (coming soon): §5 and later work. What came next, what ViT cannot do, and the whole paper on one page.

## How to read the boxes

- A **teal "From the paper" box** shows the exact lines of the paper as a highlighted screenshot (click it to open it full size). Under it, **What it says** gives the main words of the paper. **What it means** then explains those lines, and what the authors mean but do not write. When the box shows a different paper, the box gives the name of that paper.
- A **yellow definition box** explains a technical word the first time it appears. If you already know the word, skip the box.
- The small **Paper §3.1** tag above a heading tells you which section of the paper you are reading, so you can follow along in the PDF.
- A **green "Key takeaways" box** ends every part with what to remember.

Code blocks show real code, and the output shown under them is what that code actually printed. The scripts are in [`code/papers/vit`](https://github.com/ishwar6/ishwar-books/tree/main/code/papers/vit). They run on the models Google released, through the Hugging Face `transformers` library, on an Apple M5 Pro (64 GB).

## What you need to know first

Nothing about computer vision. Every term is explained when it first appears. ViT is a Transformer **encoder**, the same design as [BERT](https://ishwarj.com/papers/bert/), and the paper says so on almost every page; if you have read the BERT breakdown, Part 2 will feel familiar. If attention itself is new to you, the [attention series](https://ishwarj.com/writings/attention-1-self-attention/) explains it from zero, but the paper only needs the idea: each patch looks at every other patch.

