Lesson 1 of 4 / 7 minute read

Predicting the next piece of text

A large language model is trained on a huge amount of text to do one thing: predict what comes next. Given “The capital of France is”, it has learned that “Paris” is very likely.

Text is split into small pieces called tokens, which can be whole words or parts of words. The model produces one token, adds it to the text, and predicts again. A full answer is this loop repeated many times.

Doing this well requires the model to pick up grammar, facts, styles and reasoning patterns from its training text. That is why a prediction machine can appear so capable.

Check your understanding

What is a language model fundamentally trained to do?

Choose the answer you think is right.