AI Anatomy Class · Part 8 · The birth of ChatGPT

77. A chat is really one long document

Q. How does ChatGPT tell what I said from what it said?

1. On the chat screen, bubbles go back and forth. I ask, ChatGPT answers, and I ask again.
2. But inside the model there are no bubbles. The whole chat is joined into one long text.
3. Special marker tokens are slipped in to show who said what, like 'the user starts here' and 'the assistant starts here'.
4. At the very front goes a system prompt that users never see. It sets the role and tone the model should keep.
5. For example, it looks like this. The markers look different for each company and model, but the idea is the same.
6. The text ends with an 'assistant's turn' marker and an empty spot. The model just predicts the next piece, like in chapter 1.
7. When done, it writes the end marker and stops (chapter 2). Chapter 73's model answers had this shape, so an answer comes here.
8. The longer the chat, the longer this text. The model rereads the whole conversation every time. More in chapter 83.

Chapter summary

• A chat goes into the model as one long text, divided by marker tokens.
• The system prompt at the very front is an instruction the user can't see.
• At the 'assistant's turn' spot at the end, the model just continues with the next piece.