AI Anatomy Class · Part 1 · AI speaks in numbers

11. Matrix multiplication = transforming space

Q. What happens when you multiply a vector by a matrix?

1. Let's multiply a vector by a matrix. What comes out is a new vector.
2. We use the dot product from chapter 9. Row 1 of the matrix · the vector gives the first slot; row 2 · the vector gives the second.
3. Here's the picture. Multiply every point on the grid by the same matrix, and the whole space stretches sideways.
4. Change the matrix and the change in shape changes too. It can rotate the space, or push it over at a slant.
5. With any matrix, straight lines stay straight and the origin stays put. This kind of change is called a linear transformation.
6. GPT multiplies word vectors by matrices to reshape the space of meaning. Attention's Query, Key and Value are matrix products too.
7. Matrix multiplication is a machine that reshapes space. Choosing the numbers inside that machine is exactly what AI learning does.

Chapter summary

• Matrix × vector = new vector. Each slot is a dot product: "matrix row · vector".
• Multiply every point by the same matrix and space stretches, rotates and slants.
• Straight lines stay straight, the origin stays put (linear transformation). GPT's layers are matrix products too.