What is a large language model (LLM)?

It is a system that calculates how likely each word is to come next in a sentence. For example, after “The water in the pond is very,” words like “clear” or “cold” are likely, and “refrigerator” is not. A language model expresses this likeliness as a probability (a number for how likely something is).

When it chooses, it does not always take the word with the highest probability. The textbook explains that it usually chooses as if by a lottery in which more likely words come up more often. That is why the same question can get a differently worded answer.

Language models themselves have been around for a long time, used for things like turning spoken words into text. Early ones worked in simple ways. Today’s LLMs are far larger and more complex versions.

An LLM turns words into numbers and does lots of multiplying and adding to work out the probability of the next word. The numbers it uses for this are its parameters. Like volume knobs, turning one a little changes the result a little. But one knob does not take care of one word or one fact; many knobs are used together in the calculation.

During learning, the model is asked to guess what comes next in text, and the knobs are turned little by little in the direction that raises the probability of the right word. Repeating this over a huge amount of text gradually sets the knobs. No person turns them one by one. The amount of text it learns from is counted in “tokens” (small pieces of text).

The figure “How large is ‘large’?” above shows how much the number of knobs has grown. The textbook explains that making models bigger and training them on more text tends to improve them.

Not exactly. The LLM is the part inside a chat AI that writes the text, like its brain. ChatGPT is the name of a chat AI from a company called OpenAI. The first ChatGPT (November 2022) was made by giving a model from OpenAI’s GPT-3.5 family of LLMs further training for conversation. Claude and Gemini are names of LLMs and also of the chat AIs that use them.

An LLM that had only learned from lots of text would sometimes not answer a question, and instead write a similar question. So it is given further training, called fine-tuning, with model answers to questions and answers that people picked as better. In a 2022 OpenAI study, people preferred the answers of a fine-tuned model with 1.3 billion knobs over those of GPT-3, which has 175 billion. The researchers write that making a model bigger does not by itself make it better at doing what you ask.

Because guessing what comes next also takes knowledge. To guess what comes after “The capital of Japan is,” you need to know “Tokyo.” The textbook explains that by practicing guessing what comes next on huge amounts of text, an LLM picks up not only how words are used but also things about the world. When you ask a question, it treats the question as the start of a text and writes the continuation that is the answer.

People did not write this way of answering in, rule by rule. Anthropic, the company that makes Claude, writes that “language models like Claude aren’t programmed directly by humans—instead, they’re trained on large amounts of data.” It also writes that even its makers do not yet understand how models do most of what they do. A study by the same company also found Claude combining separate facts it knew, rather than copying out a memorized answer (for example, putting together “Dallas is in Texas” and “the capital of Texas is Austin”).

The words an LLM picks are “words that usually come next,” and those are not always right. Writing text that is factually wrong in this way, especially making up people or facts that do not exist, is called a “hallucination.”

In a 2025 paper, researchers explain that, like students guessing on hard exam questions, LLMs sometimes guess when they are unsure. The paper argues that this is because the way models are trained and scored tends to reward guessing over saying “I don’t know.”

An LLM may also not know about things that happened after it finished learning. For example, most of the training data for OpenAI’s GPT-4 (2023) ended in September 2021. An AI that can use a web search tool can look up new information and answer while showing its sources. But the same 2025 paper also writes that search reduces mistakes without getting rid of them.

Usually, just talking with an LLM does not change its parameters (knobs). It can copy examples you show it during a conversation, but according to the textbook, that stays within the conversation and does not remain inside the LLM after it ends.

The knobs change when the makers retrain it or give it extra training to make a new version of the model.

Yes, a term from law that has nothing to do with AI. LL.M. or LLM is short for Master of Laws, a degree (a title that shows what you studied) given to people who have studied law in depth at graduate school. For example, Harvard University in the United States has a one-year LL.M. program. When people talk about AI, LLM usually means a large language model.