AI Anatomy Class · Part 1 · AI speaks in numbers

9. Dot product: how much the same direction?

Q. Can one number tell how much two arrows point the same way?

  1. Here's another way to measure how alike two vectors are: the dot product. It's simple: multiply matching slots, then add it all up.
  2. Mia (8, 3) and Emma (7, 4): 8 times 7 is 56, and 3 times 4 is 12. Add them to get 68. That's the dot product.
  3. Mia and Noah (2, 9)? 8 times 2 is 16, 3 times 9 is 27, total 43. Smaller than with Emma, because their directions are less alike.
  4. Turn the arrow: the same direction gives a big dot product. A right angle gives 0. Opposite directions give a negative number.
  5. Longer arrows make the dot product bigger too. It holds both how alike the directions are and how strong the vectors are.
  6. GPT's attention uses dot products to score how related words are to each other. We'll meet it again in chapter 47.
  7. Multiply matching slots and add them all up. It's one of the calculations a transformer does most often.

Chapter summary