AI Anatomy Class · Part 5 · Mastering attention

53. Peeking inside a real GPT

Q. Are the heads of a real GPT actually 'looking' at anything?

1. At last, let's open up a real GPT: the smallest GPT-2 model. It has 12 layers with 12 heads each, 144 heads in all.
2. Feed in an English sentence and map how much one head looks at each word. Rows are the words looking; columns, the words looked at.
3. Layer 4, head 11: every word looks almost 100% at the word right before it, so only the cells just below the diagonal are dark.
4. In layer 7, head 2, almost every word looks at the first word, 'The'. Here, 98 of the 144 heads put over half on the first word.
5. Here we fed in the same sentence twice. In layer 5, head 1, the second 'my' looks 51% at 'dog', the word that followed the first 'my'.
6. In layer 4, head 3, 'it' looks 84% at 'cat'. Like the words in chapter 45, it seems to find what 'it' refers to.
7. In Part 5 we took attention completely apart: Q · K scores, √d and softmax, mixing V, matrices, the curtain, and many heads.
8. Now we know everything the attention part of the blueprint does. In Part 6 we stack the parts into a transformer block (ch. 54).

Chapter summary

• The real GPT-2 (smallest model): 12 layers × 12 heads = 144 heads.
• There really are heads that look only at the previous word, heads that pile onto the first word, and heads that copy what came before.
• Part 5 done: score (Q · K) → ÷ √d → softmax → mix V, with the curtain on and many heads at once.