AI Anatomy Class · Part 1 · AI speaks in numbers

10. Matrices: bundles of vectors

Q. Is there a way to handle many vectors at once?

1. Let's stack our three friends' taste vectors on top of each other. That gives us a table of numbers.
2. A table of numbers laid out in rows and columns like this is called a matrix. It's the same matrix you meet in high school math.
3. Lines across are rows; lines down are columns. 3 friends and 2 tastes make 3 rows and 2 columns: a 3 × 2 matrix.
4. One row is one friend's vector. One column is everyone's score for one taste. You can read the same table two ways.
5. A sentence becomes a matrix too. Each piece of "I walk to" has a vector, so stacking them gives a 3-row matrix.
6. Almost every calculation inside GPT uses matrices. In GPT-3, a matrix of (pieces × 12,288) passes through every layer.
7. A matrix is a container that holds many vectors at once. In the next chapter, we see what matrix multiplication does.

Chapter summary

• Matrix: a table of numbers in rows and columns. A stack of vectors.
• Lines across = rows, lines down = columns. Its size is "rows × columns".
• Sentence = a (pieces × vector length) matrix. Almost all of GPT's math is matrices.