Temperature and top_p both change which token a language model can select next, but they act on the probability distribution in different ways and should be tuned separately.
Temperature reshapes the whole distribution before sampling. top_p keeps the smallest set of likely tokens whose cumulative probability reaches a chosen threshold, then samples from that set. Anthropic’s current Messages API guidance says to change temperature or top_p, not both.
The short answer
| Control | What it changes | Useful when |
|---|---|---|
| Temperature | Sharpens or flattens the probability distribution | You want a broad control over variation |
top_p | Removes the low-probability tail after a cumulative cutoff | You want a probability-mass boundary |
top_k | Keeps only a fixed number of candidates | You have an advanced reason to cap candidate count |
Start with the provider’s default. If the output needs less variation, lower temperature and leave top_p unset. If you choose nucleus sampling, set top_p and leave temperature at its default. Changing both makes the result harder to diagnose because two controls are altering the same selection step.
What temperature does
A model assigns a probability to each possible next token. Temperature changes the shape of those probabilities before one token is sampled.
At a lower temperature, high-probability tokens become more dominant. The distribution gets sharper, so repeated requests are more likely to follow similar paths. At a higher temperature, probability spreads toward less likely candidates, increasing variation.
Anthropic’s Messages API currently accepts temperature from 0.0 to 1.0 and defaults to 1.0. Its reference recommends values closer to 0.0 for analytical or multiple-choice work and closer to 1.0 for creative generation.
Temperature 0 is not a proof of determinism. Anthropic explicitly notes that results are not fully deterministic even at 0.0. Backend changes, parallel computation and ties between token probabilities can still produce different outputs. Treat a low temperature as a sampling preference, not a reproducibility guarantee.
What top_p does
top_p is also called nucleus sampling. The sampler sorts candidate tokens by probability and keeps the smallest set whose cumulative probability reaches the threshold.
If top_p is 0.8, the size of that set can change from one token position to the next. A confident distribution may cross 0.8 with only a few candidates. A flatter distribution may need many. That adaptive candidate set is the main difference from top_k, which keeps a fixed number.
The cutoff is applied to probabilities, not to words, characters or a quality score. A token outside the retained nucleus cannot be selected at that step.
Why changing both is hard to reason about
Temperature reshapes the probabilities, then top_p may cut the reshaped distribution. A small temperature change can therefore change which tokens fall inside the nucleus. If both controls move between two experiments, you cannot tell which one caused the difference.
Use one control at a time:
- keep the prompt, model version and other settings fixed;
- choose temperature or
top_p; - test a small set of values on representative inputs;
- compare task quality, failure categories and output variation;
- keep a baseline using the provider’s default.
This is an evaluation problem, not a matter of finding one universal “best” number.
Temperature is not truthfulness
Lower temperature can make wording more stable, but it does not verify facts. A model can confidently repeat the same wrong answer. For extraction or application data, combine sampling choices with a schema, validation and source checks.
Our Structured Outputs vs JSON Mode guide covers format constraints. Those constraints do not make the underlying values correct either.
Likewise, prompt caching concerns repeated input work and cost. It does not make sampling deterministic or change the meaning of temperature.
When to leave the controls alone
Defaults are often the safest starting point when:
- the provider has tuned a model for a particular product surface;
- you are moving between models with different sampling behavior;
- the task already succeeds and you lack an evaluation set;
- another layer, such as a search or tool call, drives the important result;
- a reasoning model restricts or ignores sampling parameters.
Check the model and endpoint documentation. A parameter accepted by one model may be unsupported, constrained or interpreted differently by another.
A simple evaluation plan
Create a small set of inputs that represent the real task. For each setting, record:
- whether required facts or fields are present;
- whether a validator accepts the output;
- whether refusals and tool calls are handled correctly;
- how much wording varies across repeated requests;
- latency and token usage;
- the model identifier and API version.
Do not choose a setting from one attractive example. Sampling controls affect a distribution, so repeated trials matter.
For classification, extraction and strict workflows, begin with a low temperature and explicit validation. For ideation, a higher temperature may produce more varied candidates. For nucleus sampling, begin with a conservative top_p experiment and keep temperature unchanged.
Sources and version note
This guide was checked on October 31, 2026 against Anthropic’s Messages API reference, API version 2026-10-21. Parameter support and recommendations can change by model and API version; verify the current reference before deploying a fixed setting.
