LLM Temperature

Temperature is a configuration parameter that controls the degree of randomness with which a large language model (LLM) chooses each word when generating a response. When a model predicts the next token, it does not have a single option: it works with a probability distribution over many candidates. Temperature adjusts that distribution before the choice is made.

With a low temperature (close to 0), the model becomes almost deterministic: it always picks the most likely options, so it repeats the same response to the same prompt. With a high temperature (above 0.8 or 1), it flattens the distribution and lets less likely words appear, which produces more varied, creative and sometimes surprising responses.

Setting the temperature correctly is part of tuning the behavior of an automated assistant: it is one of the levers calibrated when designing AI agents for each task, whether that is extracting data precisely or writing content with style.

How it works under the hood

Every time an LLM generates text, it calculates a score for every possible token that could come next. Temperature divides those scores before they are converted into final probabilities (in practice, it scales the values fed into the softmax function). A temperature of 0 collapses everything toward the most likely token: the model becomes predictable and reproducible. As the value goes up, the gap between options shrinks, less expected tokens gain a chance and the text becomes more diverse.

Temperature should not be confused with other sampling parameters that often appear alongside it. Top-p (nucleus sampling) limits the choice to the set of tokens that together reach a certain probability; top-k restricts it to the k most likely candidates. Temperature acts on the shape of the distribution; top-p and top-k trim which candidates enter the draw. In general, you adjust one at a time so you do not lose control over the result.

Why it matters in a real project

Choosing the temperature is not a minor technical detail: it defines whether your assistant is reliable or creative. Consider an Argentine company that automates two different tasks. For an agent that classifies invoices and extracts the tax ID (CUIT), the amount and the date, you want a low temperature: you need the same invoice to always return the same data, without making anything up. For an assistant that drafts replies to customers on WhatsApp, a medium temperature (around 0.7) helps keep the messages from sounding like copies of each other and gives them a more natural tone.

A useful rule of thumb by type of task:

• Data extraction, classification, support answers based on a knowledge base: low temperature (0 to 0.3) for maximum consistency.
• Summaries and rewording: medium-low temperature (0.3 to 0.6), a balance between faithfulness and fluency.
• Idea generation, marketing copy, creative variants: higher temperature (0.7 to 1), prioritizing diversity.

Common mistakes

The most frequent mistake is raising the temperature in search of "smarter" answers. Temperature does not add knowledge or improve reasoning: it only changes how much variety it allows. Raising it too much increases the risk of hallucinations and incoherent outputs. Another mistake is expecting perfect reproducibility at temperature 0: even there, small differences can appear because of the inference infrastructure. And a third is changing temperature and top-p at the same time without measuring, which makes it impossible to know what changed the result.

Low temperature vs high temperature

In short, temperature is the most direct dial for moving a model's behavior between precision and creativity. There is no universal "correct" value: the right value depends on the task, and the best way to set it is to test with real cases from your business and measure the result, not to copy a number from a tutorial.