Tip: press / anywhere to search. Or open a random note →

Handwritten notebook page explaining how AI chatbots like ChatGPT work by predicting the next token
Tap the note to see it full size

AI & Tech One-page note

How AI Chatbots (like ChatGPT) Work

ChatGPT isn't magic. It's predicting the next word, very, very well. Here's how. Save this page.

What's on the page

The note as text, so you can read it, copy it or use a screen reader.

What are they?

  • Built on "large language models" (LLMs)
  • Examples: ChatGPT, Claude, Gemini

How they learn

  • Trained on huge amounts of text
  • Text is split into small pieces called "tokens"
  • The model learns to predict the next token

How they answer you

  • Your question is turned into tokens
  • The model predicts the most likely next token
  • Then the next... one piece at a time
  • Until a full answer appears

Why they seem smart

  • Getting very good at predicting text means learning patterns in language, facts and reasoning
  • Bigger models + more data = better predictions

Remember

  • It doesn't "look up" the truth by default
  • It can sound confident and still be wrong
  • Check important facts
Want the full story? How AI Chatbots Work: Next-Word Prediction | Scribbled Smart Read the full guide →

Sources

  1. IBM: What are large language models?
  2. OpenAI: Why language models hallucinate
  3. arXiv: A law of next-token prediction in large language models
  4. arXiv: Large language models, a mathematical formulation

How we checked this

Every fact on this note comes from the 4 sources listed here, checked against official or peer-reviewed sources where possible. AI tools help with drafting; a person checks every fact against the sources. How we research

Spotted a mistake? Tell us and we'll fix it and log it on the corrections page.

More from AI & Tech

All AI & Tech notes →