AI Anatomy Class · Part 11 · Graduation project

96. Graduation day

Q. Shall we follow the ChatGPT we assembled over 96 chapters from start to finish in one go?

1. Graduation day! Every blank in the blueprint from chapter 5 is filled in. Shall we fly through it from start to finish?
2. The tokenizer cuts 'I walk to' into pieces (chapter 29), and embedding gives each piece meaning coordinates (chapter 33).
3. Position info marks each piece's place (chapter 36). Attention lets the pieces look at each other and mix meaning (chapter 45).
4. Feed-forward adds thinking to each piece (chapter 55), and the block is stacked dozens of layers high. GPT-3 has 96.
5. The last vector becomes scores for about 50,000 tokens, and softmax turns them into odds (chapters 60, 14): school 52%, the 21%.
6. Pick one, append it, back to the top (chapter 2). Pre-training set the knobs; model answers and praise taught it to chat.
7. And you trained a mini GPT on this same blueprint yourself. Only the size is different; it's the same machine.
8. You've finished all 96 chapters of AI Anatomy Class. Now you can explain how ChatGPT answers. Congratulations!

Chapter summary

• Tokenizer → embedding + position → [attention + feed-forward] × N → output + softmax → pick and append.
• Pre-training sets the knobs; chat practice with model answers and praise teaches it to talk.
• I trained a mini GPT on the same blueprint. The only difference is size.