SAM, read beside the paper

The paper and its explanation side by side.

• Part 1: The Big Idea: Segment Anything

  Meet SAM, a model that cuts one object out of a photo after one click. We read the first two pages of the paper, and we run SAM on a photo of two cats.

• Part 2: The Task: Promptable Segmentation

  What SAM must learn: a valid mask for any prompt. We read §2 and Figure 3 of the paper, and we test ambiguous clicks, extra points, boxes and the paper's training game on the cat photo.

• Part 3: The Model: Encoder, Prompts and Decoder

  How SAM turns a photo and a click into three masks and three scores. We open the released model, follow each tensor through it, and compute its training loss by hand.