AI Anatomy Class · Part 6 · Assembling the transformer

62. Part 6 boss battle: the finished blueprint

Q. We've collected every part. Can you assemble them in the right order and complete GPT?

1. In Part 6 we collected the block's parts one by one. Now let's put them all together and complete the blueprint!
2. Start with one block layer: normalize → attention → add, then normalize → feed-forward → add.
3. Stack this block 96 layers deep. Thanks to the highway and normalization, it stays steady even that deep.
4. In front, the tokenizer, embedding and position info turn the text into vectors and feed them in (Part 3).
5. At the back, output + softmax gives the next word's probabilities, and one word is picked and appended.
6. The machine part of the blueprint is complete! It's a GPT that takes in text and gives out the next word.
7. But the 175 billion numbers inside its parts are still a mess. Part 7 teaches them. That's called pre-training.

Chapter summary

• One block layer: normalize → attention → add → normalize → feed-forward → add. Stack it 96 times.
• Blueprint: tokenizer → embedding → position info → block × 96 → output + softmax → pick and append.
• The machine is complete! Next comes pre-training, which teaches its 175 billion numbers (Part 7).