AI Anatomy Class · Part 2 · How machines learn

16. A single neuron

Q. A new taco truck just opened. How would an AI decide whether it's a hit?

1. A nerve cell in your brain (a neuron) collects signals from many places. If the total is strong enough, it sends a signal on.
2. Let's copy that idea to judge taco trucks. Truck A: spiciness level 3, 2 people in line, $4 a taco.
3. Each input is multiplied by a weight: a knob for how much that info matters (chapter 12). A negative weight takes points off.
4. Add up the products, then add the bias, 2: the base score when there's no info at all. The total is 3.5 points.
5. We said 4 points or more means a hit. It's 3.5, so this neuron decides: not a hit.
6. Turn a knob and the decision changes. Raise the line weight from 1 to 1.5 and the score is 4.5. Now it's a hit!
7. The 1958 perceptron was a neuron like this (chapter 3). Most of GPT-3's 175 billion knobs are weights like these, too.
8. Multiply, add, add the bias, then decide: that's one neuron. But this neuron has a weakness.

Chapter summary

• Neuron: multiply each input by its weight, add them all up, then add the bias.
• Weight = a knob for how much the info matters (negative = points off). Bias = base score.
• Change the knobs and the decision changes. Learning = finding good knobs.