What is Temperature?

At each step a model produces a probability distribution over possible next tokens. Temperature reshapes that distribution before one is sampled: low values sharpen it toward the top candidate, making output near-deterministic and repetitive; high values flatten it, letting unlikelier tokens through, which reads as creative or, past a point, incoherent. Typical practice is near 0 for extraction, classification and tool calls, and 0.7 to 1.0 for drafting. It isn't an accuracy dial — a low temperature will state a wrong fact very consistently.

Two things people get wrong. Temperature 0 is not a guarantee of identical output — batching, hardware and provider-side details still introduce variation. And it isn't a single global setting for an agent: the right value differs per step. A Velaris agent that routes across models is already choosing an engine per task; sampling is the same idea one level down — a step that emits a structured tool call wants determinism, while a step drafting the email that tool will send wants room to move.