AI Anatomy Class · Part 9 · The moment an answer is made

84. Convincing lies

Q. If an AI says something confidently, is it true?

1. We asked about a book that doesn't exist, and it answered smoothly, with a year and even a prize. That's called a hallucination.
2. Let's replay it piece by piece. Book intros often start with a year, so 'A 2018' sounds likely. 'I can't find' gets only 4%.
3. Same for the next pieces: fairy tale, that won, a children's. Each pick is plausible, but together they make a fake fact.
4. What the model learned is 'the word likely to come next' (chapter 1). Nothing in it checks whether that word is true.
5. And a written word can't be erased (chapter 2). Once 'A 2018' is written, everything after it follows along plausibly.
6. Hallucinations are most common with rare facts, numbers and dates, names, and sources. It can even invent real-looking sources.
7. So check sources yourself when it matters. Giving it reliable material makes it invent much less (chapter 88).
8. What about lowering the temperature? Odd picks drop, but if the top answer is made up, it stays. Plausible isn't the same as true.

Chapter summary

• Hallucination: making up something untrue that sounds convincing.
• The model only picks a likely next word. It doesn't check whether that word is true.
• Check numbers, names and sources yourself; giving it material makes it invent less (ch. 88).