On this page
AI chatbots like ChatGPT work by predicting text: they write an answer one small piece at a time, each time choosing a likely next piece based on what came before. That is the whole trick, done extremely well. This guide explains how AI chatbots work in plain English, from training to the answer on your screen, and why you should still check what they tell you.
What is an AI chatbot?
An AI chatbot is a program you can type to, and it types back in normal language. Well-known examples are ChatGPT (made by OpenAI), Claude (made by Anthropic) and Gemini (made by Google).
Under the hood, each one is built on a large language model, usually shortened to LLM. An LLM is a computer model that has learned the patterns of language from a very large amount of text. The chat window is just the friendly front door to it.
How do AI chatbots learn from huge amounts of text?
Before a chatbot can talk to you, its language model goes through training. During this stage, called pretraining, the model is shown huge amounts of text and practises one job again and again: guessing what comes next.
Text is split into tokens
Computers don't read words the way we do. So the text is first chopped into small pieces called tokens. A token can be a whole word or a part of a word.
For example, a short common word might be one token, while a long or unusual word might be split into a few pieces. Everything the model reads, and everything it writes, is made of these tokens.
The model learns to predict the next token
The model sees a stretch of tokens and tries to predict the next one. It practises this over a huge amount of text, and becomes very good at predicting what comes next.
Several sources describe this the same way, including IBM's explainer, OpenAI's own research and academic papers on next-token prediction.
Training after training
The note keeps things simple, but there is one more step worth knowing. After pretraining, chatbots are usually fine-tuned so they follow instructions and behave like helpful assistants. One method is instruction tuning, which IBM describes as improving a model's ability to follow human instructions. Another is RLHF (reinforcement learning from human feedback): people rank the model's answers, and the model is trained to prefer the ones people ranked higher.
How AI chatbots work when you ask a question
When you type a message, this is roughly what happens:
- Your message is turned into tokens.
- The tokens go through the model, which works out how likely each possible next token is.
- One token is selected.
- That new token is added to the text, and the model predicts the next one.
- This repeats, one piece at a time, until a full answer appears.
So when you watch an answer appear word by word on screen, you are seeing something close to how it is really made. The chatbot is not pulling out a finished paragraph. It is building it one token after another.
Transformers and attention
Large language models are built on a design called the transformer, introduced in a 2017 paper by Vaswani and colleagues called "Attention Is All You Need". Its key idea is self-attention: as the model handles each token, it weighs how much every other token in the text matters to it. That helps it keep track of meaning across a whole sentence or paragraph.
The context window
There is a limit to how much text a model can consider at once. This is called the context window, and it is measured in tokens. Anything outside that window, the model can't take into account.
Why do AI chatbots seem so smart?
If a chatbot only predicts the next token, why does it seem to understand things? Because getting very good at predicting text means picking up the patterns in that text. Those patterns include grammar, facts that appear often, and steps of reasoning that people write down.
Sources describe abilities like following instructions and reasoning as side effects of that one training goal. In other words: to predict text really well, the model has to learn a lot about how language and ideas fit together.
Bigger models and more data
Researchers have found that performance tends to improve in a predictable way as models get more parameters (the model's internal settings), more data and more computing power. This is often called scaling.
For a sense of size, GPT-3 had 175 billion parameters when it was released in 2020. Newer models don't always say how big they are.
What to remember: it doesn't look up the truth
This is the most important part. By default, a chatbot does not look up the truth. It predicts what text is likely to come next. That can give a good answer, but sometimes the most likely-sounding text is simply wrong.
OpenAI's own research explains that these mistakes, often called hallucinations, come from how models are trained and how they are tested. We cover that in detail in our guide to why AI makes things up.
Note the words "by default". Many chatbots can now search the web or use other tools, which can help. But the core of how they write is still prediction.
Tips and common mistakes
- ✓ Check important facts. Names, numbers, dates and quotes are worth confirming somewhere reliable.
- ✓ Give clear instructions. The better your prompt, the better its prediction of what you want.
- ✗ Trusting the tone. A chatbot can sound confident and still be wrong. Confidence comes from fluent language, not from proof.
- ✗ Treating it like a search engine. Unless it is actually searching, it is writing from patterns it learned, not reading a live source.
Related reading
- Why AI makes things up: the next step, explaining why chatbots give confident wrong answers.
- How to write a good AI prompt: turn what you now know into better answers.
- How the internet works: another everyday technology explained from the ground up.
- Cognitive biases: why we tend to trust things that sound confident.
Sources
- IBM: What are large language models?
- OpenAI: Why language models hallucinate
- arXiv: A law of next-token prediction in large language models
- arXiv: Large language models, a mathematical formulation
How we checked this
Every fact in this article comes from the 4 sources listed above, checked against official or peer-reviewed sources where possible. AI tools help with drafting; a person checks every fact against the sources. How we research
- Published and checked
- Next check
- Yearly (fast-moving field)
General info. AI tools change fast, so check each product's own help pages for its current features.
Spotted a mistake? Tell us and we'll fix it and log it on the corrections page.