AI Anatomy Class · Part 2 · How machines learn

19. The forward pass

Q. When you put numbers into a neural network, what happens inside?

1. Let's feed Truck A into a small layered network: 3 inputs, a layer of 2 neurons, and one output neuron that gives a rating.
2. The input numbers flow along the lines and get multiplied by weights. The "popular" neuron: 1.5 + 2 − 2, plus bias −0.5, gives 1.
3. The "too spicy" neuron is an alarm that turns on above spiciness level 4. 3 − 4 = −1, so ReLU switches it off to 0.
4. The layer's outputs, 1 and 0, flow on to the next layer: 1 × 1.5 + 0 × (−1) + 2 = 3.5. The predicted rating is 3.5.
5. Computing step by step in one direction, from input to output, is called the forward pass.
6. At spiciness level 5, the alarm neuron turns on and takes 1 point off. Different inputs make different neurons do the work.
7. GPT answers by a forward pass too: text runs through 96 layers in turn, and out come next-word probabilities. But was 3.5 right?

Chapter summary

• Forward pass: input → layer → output, computed step by step in one direction.
• Each neuron: sum of (input × weight) + bias, then bend it with ReLU.
• GPT answers with a forward pass too: through 96 layers to next-word probabilities.