BroadRouter
Get started

max_tokens

Updated

max_tokens is an LLM API parameter that sets the most tokens a model may generate in one response. When generation reaches the limit, it stops and the response is marked as cut off, for example with finish_reason set to "length".

Also called: max_completion_tokens, max_output_tokens, output token limit, maximum output length

max_tokens is the output budget for one reply. It protects you from runaway answers, but set too low it cuts answers off or returns nothing at all.

How it works

The model generates one token at a time. When the count reaches max_tokens, generation stops, even mid-sentence. The response then says why it ended:

finish_reason Meaning
stop The model finished on its own
length The max_tokens limit was reached
tool_calls The model is asking to call a tool

The parameter has several names. OpenAI’s Chat Completions API now marks max_tokens as deprecated in favor of max_completion_tokens, an upper bound that includes both visible output and reasoning tokens. OpenAI’s Responses API uses max_output_tokens. Anthropic’s Messages API uses max_tokens. Many OpenAI-compatible providers accept max_tokens.

The prompt and the output share one context window. If the window is 32,768 tokens and the prompt is 30,000, at most 2,768 tokens can come back, whatever max_tokens says.

Why it matters

  • Cost control. On a per-token key, max_tokens is the most a single reply can cost in output.
  • Empty answers. On reasoning models, thinking comes out of the same budget. A limit of 100 can be spent entirely on reasoning tokens, so the visible answer is empty and finish_reason is length.
  • Broken JSON. A cut-off reply leaves structured output incomplete and unparseable. Check finish_reason before you parse.

On BroadNet Inference

DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before answering, and BroadNet’s docs say the reasoning counts toward max_tokens. Allow 1,000 tokens or more, leave it unset, or lower reasoning_effort:

response = client.chat.completions.create(
    model="broadnet/glm-5.3",
    max_tokens=2000,
    messages=[{"role": "user", "content": "Summarize this contract clause."}],
)
if response.choices[0].finish_reason == "length":
    print("Cut off: raise max_tokens or lower reasoning_effort")

Context sizes differ by model: 1,048,576 tokens for DeepSeek V4.1 Flash and GLM 5.3, 262,144 for Qwen3.8 27B and 32,768 for BroadNet Gemma 4 VL 26B. See the model pages and why reasoning models return empty answers.

Common mistakes

  • Copying a small limit from a non-reasoning model during a migration. See the migration checklist.
  • Treating length as success. Log it and retry with more room.

Questions

Is max_tokens the same as the context window?

No. The context window is the total the model can handle, prompt plus output. max_tokens limits only the output, and prompt plus max_tokens must still fit in the window.

Should I always set max_tokens?

Set it when you need a hard cap on cost or length. Leave generous room on reasoning models, or leave it unset if the provider's default is large enough.

What is the difference between max_tokens and max_completion_tokens?

OpenAI deprecated max_tokens on Chat Completions in favor of max_completion_tokens, which explicitly includes reasoning tokens. Many OpenAI-compatible providers still accept max_tokens.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity