AI Anatomy Class · Part 5 · Mastering attention

46. Query · Key · Value

Q. How does a word decide which other words in the sentence to look at?

1. In chapter 45, 'trunk' looked more at related words. Measuring 'related' works a lot like finding a book in a library.
2. At the library, you start with something you want: 'Any stories about school?' That question card is the Query.
3. Each book has a label on its spine. You match your question against the labels and pick the books that fit. The label is the Key.
4. Open the book you picked and there's its content: the information you actually take. That's the Value. Pick by label, take the content.
5. Words in a sentence work the same way. Every word holds three cards: Q to ask, K to say what it is, and V to hand over.
6. 'school' asks something like 'Is anything headed toward me?' That fits the label on 'to' best, so it takes the most of 'to''s content.
7. The cards are made by multiplying the word vector by matrices (ch. 11). School's vector (1, 1) times three matrices gives Q, K and V.
8. The numbers in the matrices are learned. How well a question fits a label, we'll measure next chapter with chapter 9's dot product.

Chapter summary

• Query is what I'm looking for, Key is the label that says what I am, Value is the content to hand over.
• Match questions against labels to decide whom to look at, then take that much of their content.
• Q, K, V = matrix × word vector (ch. 11). The numbers in the matrices are learned.