Lesson 2 of 4 / 5 minute read

Why answers change each time

At each step the model has a list of possible next tokens with probabilities. It usually does not always pick the top one. A little randomness makes writing more natural and varied.

That is why asking the same question twice can produce two different answers. Neither is “the” answer stored somewhere. Both were generated fresh.

For creative work this is useful. For facts it is a warning: if two runs disagree about a date or a number, at least one is wrong and you need another source.

Check your understanding

Two runs of the same question give different facts. What should you do?

Choose the answer you think is right.