AI Anatomy Class · Part 8 · The birth of ChatGPT

72. Smart but doesn't listen

Q. Ask a question to a model that has read a huge amount of internet text. Will it simply answer?

1. Here's a model that finished Part 7: it learned next-piece guessing from internet text. It's called a base model.
2. Let's ask it: 'What's France's capital?' Surely it will answer Paris, right?
3. Instead of answering, it keeps writing questions: Italy's capital? Spain's capital? Like a list of quiz questions.
4. To the model, it looked like the first line of a quiz. Online, questions are often followed by more questions.
5. All a base model learned is to continue text plausibly (chapter 2). It was never taught to answer questions.
6. So people used a trick: start with 'Question: … Answer:', and the next piece is far more likely to be an answer.
7. But you can't use a trick every time. Teaching this smart but disobedient model to listen is what Part 8 is about.

Chapter summary

• A base model has only learned to guess the next piece of internet text.
• Give it a question, and instead of answering it may keep writing questions, like a workbook.
• Changing the shape of the text can draw out an answer, but the real fix is to teach it separately.