AI Anatomy Class · Part 1 · AI speaks in numbers

12. Functions are input→output machines

Q. People say an AI model is really just a "function". What does that mean?

1. A machine that takes a number, changes it by a fixed rule and sends it out: that's a function.
2. If f(x) = 2x + 1, putting in 3 gives 7. Putting in 5 gives 11. The same input always gives the same output.
3. Functions can be chained. Put f's output, 7, into g, which squares it, and you get 49. This is called composition.
4. Functions have knobs inside. Change the 2 and the 1 in 2x + 1, and the machine behaves differently. These knobs are parameters.
5. GPT is a giant function too. Text goes in; next-piece probabilities come out. Inside, hundreds of small functions are chained.
6. GPT-3 has 175 billion knobs. Learning means turning these knobs bit by bit until the answers come out right.
7. An AI model is a function with a huge number of knobs. In Part 2, we learn how to turn them.

Chapter summary

• Function: a machine that changes an input by a rule into an output. Same input → same output.
• Functions can be chained (composition). The knobs inside are parameters.
• GPT = a function with 175 billion parameters. Learning = setting the knobs.