How LLMs Work: Build One Small Enough to Understand

Complete this sentence before reading on:

The coffee is ___

You might choose hot, ready, cold, or good. You did not search a dictionary for the answer. You used the words around the blank and your experience of how people write.

A large language model, or LLM, begins with a related job: given the text so far, what small piece of text is most likely to come next?

That sounds too simple to explain a chatbot, a code assistant, or a writing tool. The important word is repeatedly. A model predicts one piece, adds it to what it can see, predicts the next piece, and continues.

The coffee is
        ↓
The coffee is hot
        ↓
The coffee is hot and
        ↓
The coffee is hot and ready

Modern LLMs have far more data, much larger neural networks, and more capable ways to use context than the example above. But this loop is the durable mental model:

Read the context → score possible next tokens → choose one → repeat

We will build the smallest version of that idea first. Then we will see exactly what modern systems add and why they can still sound convincing when they are wrong.

Start with a model that only remembers neighbours

Imagine a very small training collection:

the cat sat on the mat
the cat ate the fish
the dog sat on the mat

If this model sees the word cat, it can notice that sat followed it once and ate followed it once. If it sees on, it has only seen the next. It can make a guess by counting what appeared after each word.

const trainingText = `
the cat sat on the mat
the cat ate the fish
the dog sat on the mat
`;

const words = trainingText.trim().split(/\s+/);
const nextWordCounts = new Map();

for (let index = 0; index < words.length - 1; index += 1) {
  const word = words[index];
  const nextWord = words[index + 1];
  const counts = nextWordCounts.get(word) ?? new Map();

  counts.set(nextWord, (counts.get(nextWord) ?? 0) + 1);
  nextWordCounts.set(word, counts);
}

console.log(nextWordCounts.get("cat"));
// Map { "sat" => 1, "ate" => 1 }

This is a language model in the loosest sense. It learned a pattern from text and uses that pattern to predict a continuation. It is also extremely limited. It does not know that cat and dog are both animals. It cannot use the word at the start of a long sentence. It only remembers one neighbour at a time.

Add this line to the training text and run it again:

the fish swam away

Now the has several possible continuations. That is the first useful discovery. A language model does not usually have one guaranteed answer. It has a set of possibilities shaped by what it learned.

A model reads tokens, not words

The tiny predictor above splits on spaces because that is easy for us to read. Real LLMs cannot safely treat every whole word as one unit. There are too many words, names, spellings, programming symbols, and languages.

Instead, an LLM first divides text into tokens. A token is a small piece of text. Sometimes it is a whole word. Sometimes it is part of a word, a space attached to a word, punctuation, or a code symbol.

"unhappiness" might become [ "un", "happi", "ness" ]
"console.log" might become [ "console", ".", "log" ]

Each token maps to an ID the model can process:

"The"   →  464
" coffee" →  6891
" is"    →  318

The exact IDs do not carry meaning by themselves. They are labels, like seat numbers in a theatre. The useful part comes next, when the model turns those labels into learned numerical representations.

Tokens matter for three practical reasons. They determine how much text fits in the model's context window, how API usage is measured, and why two sentences with a similar number of characters can cost different amounts to process.

Here is a prediction: if you replace one familiar English word with a very unusual identifier from a codebase, will it always count as one token?

No. A tokenizer may need several pieces for an unfamiliar string. That is why code, long URLs, and some languages can use more tokens than a reader expects.

From counts to probabilities

Our tiny model can count which word came next. A real language model produces a probability for every token it could choose.

Context: "The coffee is"

hot       0.42
cold      0.18
ready     0.16
good      0.11
empty     0.03
everything else  0.10

The model may choose the highest probability every time. That makes output predictable, but it can make writing repetitive. Or it can sometimes sample a less likely option. That is where a setting such as temperature has an effect: lower temperature concentrates choices near the most likely token, while higher temperature gives less likely tokens more chance.

Probability is not truth. It answers, "what continuation fits the patterns this model learned?" It does not answer, "what happened in the world?"

For example, a model might confidently continue an invented book title with a plausible author name. The sentence can fit patterns of book citations even when the book does not exist. This is one reason LLM output needs verification when facts matter.

Training changes the model. Chatting uses it.

Two different moments are often mixed together: training and generation.

During training, the model sees text where the next token is already known. It makes a prediction, compares it with the actual next token, and measures how wrong it was. A mathematical process adjusts millions or billions of values called weights so that future predictions become a little better.

Training example: "The capital of France is Paris"

Context: "The capital of France is"
Model prediction: "London"
Correct next token: "Paris"
        ↓
Adjust the weights so "Paris" receives a higher score next time

Weights are not sentences stored in a hidden folder. They are the numbers that shape how the network transforms input into a prediction. After enormous numbers of examples, those values encode useful statistical patterns: grammar, common facts, styles of explanation, relationships in code, and many others.

When you chat with an LLM, normal generation does not rewrite those weights for your conversation. It uses the weights it already has and the text currently in the prompt.

Parameters  = patterns learned during training
Context     = text the model can see right now

This distinction explains a common surprise. Tell a model, "My project uses PostgreSQL," and it can use that fact in the current conversation. The fact is in its context. Start a fresh conversation, and it may not know it unless the application supplies it again.

Numbers can capture useful relationships

The token ID 464 for The is only a label. To reason about token relationships, the model converts every token into a list of learned numbers called an embedding.

You can imagine a tiny embedding as coordinates on a map:

cat  → [ 0.2,  0.8, -0.1 ]
dog  → [ 0.3,  0.7, -0.2 ]
SQL  → [-0.6,  0.1,  0.9 ]

The real lists are far longer. The point is not that one number means "animal" and another means "database." Meaning is spread across many dimensions. But during training, tokens used in similar situations can end up with representations that make useful relationships easier for the network to detect.

Our word-count model treated cat and dog as unrelated labels. A neural network can learn that patterns involving them often behave similarly. That makes it better at handling text it has not seen in exactly the same form.

Long context creates a new problem

Suppose the model sees this sentence:

Maya placed the blue mug beside the red notebook because it was still wet.

What does it refer to? The notebook is nearer, but the phrase "still wet" makes the mug a stronger candidate. A model that only checks the previous word has no chance. It needs to compare the current position with earlier parts of the sequence.

Attention is the mechanism that lets a transformer decide which earlier tokens deserve focus for the current prediction. It does not look at every earlier token with equal importance. It calculates relevance scores and combines information accordingly.

Before looking at how a model scores this, make your own prediction. Which earlier phrase should receive the strongest focus for it?

• blue mug
• red notebook
• Maya

Here is a simplified picture of how attention weights that relationship:

When predicting the meaning of "it":

blue mug       █████████  strongest focus: "still wet" fits a mug
red notebook   ██         weaker focus
Maya           █          little focus

This is a teaching picture, not a literal trace from a real transformer. Attention has many heads and layers, and different heads can learn different kinds of relationships. One may help follow grammar. Another may help connect a pronoun to a noun. Another may help track a repeated technical term.

A transformer is the engine for that work

Most modern LLMs use a neural-network architecture called a transformer. The name matters less than the sequence of jobs it repeats across many layers:

Tokens
  ↓
Embeddings add learned numerical representations
  ↓
Attention shares relevant context between positions
  ↓
Neural-network layers transform that information
  ↓
Scores for the next token

Each layer refines the representation. Early layers may capture local patterns. Later layers can combine more abstract information. At the end, the model assigns a score to every possible next token and turns those scores into probabilities.

The transformer does not contain a little person reading a sentence. It is a very large mathematical system trained to make useful next-token predictions. Yet at enough scale, that objective produces behaviour that looks surprisingly capable: following instructions, translating, explaining code, summarising, and writing in different styles.

Capability does not remove limits. A fluent explanation can still omit context, reflect mistakes in training material, or invent a fact. Treating fluent language as proof is the mistake to avoid.

One token at a time becomes an answer

When an assistant answers you, it does not usually create the final paragraph in one move. It runs a generation loop:

Prompt: "Explain DNS in one sentence."
        ↓
Choose: "DNS"
        ↓
Choose: " maps"
        ↓
Choose: " domain"
        ↓
Choose: " names"
        ↓
Continue until a stopping token or length limit

Every selected token becomes part of the next context. That means one early choice can change the rest of an answer. It also explains why a clearer prompt often changes the result. You are changing the context from which every next prediction is made.

This is also why a long conversation can be slower and more expensive. The model has more context to process, and it must produce the response token by token.

When a convincing answer needs checking

Imagine you ask an assistant how to use a library, and it gives you this code:

import { fancyFunction } from "useful-package";

fancyFunction();

The import looks ordinary. The function name fits the package. But your editor says that fancyFunction does not exist. The model did not inspect your installed package unless the application gave it that information. It continued a pattern that looked likely: package names are often followed by a named import and a plausible function call.

The useful response is not "the model lied" or "AI cannot write code." Check the package documentation, your installed version, and the exact error. Then give the model the evidence: "Version 4 is installed, and this export is missing. Use the official API instead." A language model can help you reason through the next step, but fluent code is not proof that an API exists.

A chatbot is more than the base model

The base model is the prediction engine. A useful assistant often adds more layers around it:

Base model          predicts the next token
Instruction tuning  makes it better at following requests
Safety rules        restrict harmful or unwanted behaviour
Retrieval           supplies current documents or search results
Tools               let it call APIs, run code, or fetch data
Application state   supplies conversation history and product context

Retrieval is particularly important. A base model does not look up today's weather, your private database, or a newly published Devloom tutorial by itself. An application can search a trusted source, put the relevant results into the prompt, and ask the model to answer using that supplied material. That is the basic idea behind retrieval-augmented generation, often shortened to RAG.

Fine-tuning is different. Fine-tuning changes weights through more training. Retrieval changes the information available in the current context. If you need a model to know a current company policy, retrieval is usually the right first move. If you need a durable change in how a model behaves across many requests, fine-tuning may be worth considering.

The model you can now explain

An LLM is not a fact database and it is not a person. It is a neural network trained on huge amounts of text to predict the next token. During a response, it reads the tokens in its context, uses learned weights and attention to score possible next tokens, chooses one, and repeats.

That single loop is not the whole story, but it is the part that keeps the rest understandable. Tokens make text usable by the model. Training creates the learned weights. Embeddings and attention help the network use relationships and context. Retrieval and tools can give an assistant information the base model did not already have.

The next time an LLM produces a polished answer, you can ask a better question than "how did it know that?" Ask: what context did it receive, what patterns did its training make likely, and where did the factual information come from? That is how you use language models with both curiosity and judgment.