AI Anatomy Class · Part 8 · The birth of ChatGPT

73. Copying model answers

Q. How do you teach a model that doesn't listen how to answer?

1. The fix is surprisingly simple. People write questions and model answers themselves and show them to the model.
2. Translation, math, poems, explanations: for each question they write a kind, accurate answer, and collect lots of them.
3. Training works just like Part 7: guessing the next piece. Only the textbook changes, from internet text to model answers.
4. The model is already smart, so its numbers only need a light touch. This is called supervised fine-tuning (SFT).
5. The result is amazing. Asked the same question, it no longer writes more questions. It answers properly.
6. It even answers new questions. It didn't memorize answers; it learned the shape and tone: after a question comes an answer.
7. But writing model answers is slow, hard work. So people found an easier way. See the next chapter.

Chapter summary

• SFT: train the model a little more on questions and model answers written by people.
• The method is the same next-piece guessing as pre-training; only the textbook changes.
• The model doesn't memorize answers; it copies the shape and tone of answering.