Temperature is a sampling parameter that controls randomness in a language model's output by scaling the probability distribution before the next token is chosen.
Temperature divides the logits by a scalar before softmax. Low temperature sharpens the distribution, making high-probability tokens even more likely. High temperature flattens it, giving weaker tokens a better chance and producing more varied output.
That single knob changes output style without touching the model weights. A model answering medical questions should run near zero temperature so it stays consistent and safe. A model brainstorming marketing taglines benefits from higher temperature so it explores more alternatives. Temperature does not make the model smarter or dumber; it trades determinism for diversity.
Think of it like this. Think of a DJ selecting the next track. Low temperature plays the obvious crowd favorite every time. High temperature pulls deeper cuts and surprises. Both are valid; it depends on whether the room wants comfort or discovery.
After the model produces raw logits for every token in the vocabulary, temperature divides each logit by the temperature value. The result is passed through softmax to create a new probability distribution. Temperature 1.0 leaves logits unchanged. Below 1.0 sharpens the distribution; above 1.0 flattens it.
"Higher temperature makes the model creative." It makes output more random, which sometimes reads as creative. "Temperature controls knowledge." It only controls which token is sampled, not what the model knows. "Temperature 0 is always best." It removes randomness entirely, which is ideal for factual QA but bad for brainstorming or open-ended chat.
Low temperature gives predictable, focused answers. High temperature gives variety but increases the chance of hallucinated or inconsistent responses. Production systems often use different temperatures per feature rather than one global setting.