---
title: "What is top-p? Nucleus sampling in LLMs explained | BroadRouter"
description: "Top-p, or nucleus sampling, samples only from the smallest set of tokens whose probabilities add up to p. How it works and how it differs from temperature."
url: https://broadrouter.com/glossary/top-p/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Top-p (nucleus sampling)

Updated 2026-11-14

Top-p, also called nucleus sampling, is a decoding method in which a language model samples the next token only from the smallest set of most likely tokens whose combined probability reaches a threshold p, such as 0.9, and discards the unlikely tail.

Also called: nucleus sampling, top\_p, top-p sampling

Top-p decides which tokens are allowed into the draw before the model picks one.

## How it works

Holtzman and colleagues introduced nucleus sampling in 2019. They define the top-p vocabulary as the smallest set of tokens whose cumulative probability is at least p. Every token outside that set gets probability zero. The rest are rescaled to sum to one, and the next token is sampled from them. Their main results use p = 0.95.

The set changes size at every step:

| Next-token distribution | Tokens kept at p = 0.9 |
| --- | --- |
| One token at 0.95 | 1 |
| Four tokens at 0.3, 0.3, 0.2, 0.15 | 4 |
| A long flat spread | many |

That is the point of the method. The paper argues the “less reliable tail” of the distribution causes degenerate text, and that cutting it gives varied output without losing coherence. A fixed top-k cannot adapt this way: it keeps the same count whether the model is certain or unsure.

In OpenAI’s API, `top_p` defaults to 1, meaning no cut. The reference gives the example that “0.1 means only the tokens comprising the top 10% probability mass are considered.” Hugging Face Transformers also defaults to 1.0 unless the model’s `generation_config.json` sets another value.

## Why it matters

-   **Fewer strange tokens.** Cutting the tail removes the rare picks that make text drift, which matters for long answers and for [hallucination](https://broadrouter.com/glossary/hallucination/) risk.
-   **Interaction with temperature.** [Temperature](https://broadrouter.com/glossary/temperature/) is applied to the scores; top-p then trims the result. Tuning both at once makes behaviour hard to predict, which is why OpenAI suggests changing only one.
-   **Debugging.** If [logprobs](https://broadrouter.com/glossary/logprobs/) show the model chose a low-probability [token](https://broadrouter.com/glossary/token/), a lower top-p would have blocked it.

Top-p does not change price. Every generated token is billed the same.

## On BroadNet Inference

BroadNet’s [API reference](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-top-p&utm_content=body) does not document `top_p`, `top_k` or their defaults. Its request examples set neither. An OpenAI SDK will pass `top_p` through, but whether a given model applies it is not stated, so test on your own prompts before relying on it. See the [Python SDK guide](https://broadrouter.com/guides/openai-python-sdk/) and the post on [evaluating a model before switching](https://broadrouter.com/blog/evaluate-llm-before-switching/), which explains why sampling settings must be fixed during a comparison.

## Common mistakes

-   Tuning `top_p` and temperature together, then not knowing which one changed the result.
-   Copying sampling settings from one model to another. Recommended values differ by model.

## Questions

### What is the difference between top-p and temperature?

Temperature reshapes all token probabilities. Top-p cuts off the unlikely tail and samples from what is left. OpenAI recommends changing one or the other, not both.

### What is the difference between top-p and top-k?

Top-k keeps a fixed number of tokens, such as 50. Top-p keeps however many tokens it takes to reach the probability threshold, so the set is small when the model is sure and large when it is not.

### Does BroadNet document top\_p?

No. BroadNet's API reference does not list top\_p or its default for any model. Set it in the request and test the effect yourself.

## See also

-   [Temperature](https://broadrouter.com/glossary/temperature/) (term)
-   [Logprobs](https://broadrouter.com/glossary/logprobs/) (term)
-   [Token](https://broadrouter.com/glossary/token/) (term)
-   [Hallucination](https://broadrouter.com/glossary/hallucination/) (term)
-   [Frequency and presence penalty](https://broadrouter.com/glossary/frequency-and-presence-penalty/) (term)
-   [Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/) (guide)
-   [How to evaluate an LLM before switching providers](https://broadrouter.com/blog/evaluate-llm-before-switching/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-top-p&utm_content=page-cta) ·

**Sources**

-   [Holtzman et al. (2019), The Curious Case of Neural Text Degeneration](https://arxiv.org/abs/1904.09751)
-   [OpenAI API reference, create chat completion (top\_p)](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create)
-   [Hugging Face Transformers, GenerationConfig (top\_p, top\_k)](https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
