AI Anatomy Class · Part 1 · AI speaks in numbers

13. Probability distributions

Q. How would you say "it'll probably rain" with numbers?

1. Roll a die and each number from 1 to 6 comes up with probability 1/6. That's the probability you learned in middle school.
2. Draw a bar for each outcome's probability and you get a probability distribution. The bar heights always add up to 1, or 100%.
3. Not every distribution is even. For breakfast, cereal is high and pizza is low. We call this a peaked distribution.
4. GPT produces a distribution like this every moment: about 50,000 candidate pieces, so 50,000 bars. They still add up to exactly 1.
5. When you draw one from a distribution, taller bars get picked more often. But lower bars get picked sometimes too.
6. A peaked distribution gives steady answers; a flat one gives varied answers. In chapter 81 we control this with temperature.
7. A probability distribution gives every candidate a chance, and they add up to 1. In chapter 14, we turn scores into one.

Chapter summary

• Probability distribution: a chance for every candidate. All the bars add up to 1.
• Taller bars get picked more often, but lower bars get picked sometimes too.
• Every time, GPT makes a distribution over about 50,000 candidates and picks one.