A reasoning model thinks before it answers, and on most OpenAI-compatible APIs both come out of one budget: max_tokens. Set it the way you would for a non-reasoning model, a few hundred tokens for a short answer, and some replies come back empty with finish_reason: "length". The quick fix is in why a reasoning model returns an empty answer. This post is the method behind a budget you can trust: measure, size from percentiles, respect the context window, and handle the replies that still get cut off.
For what reasoning costs and how to reduce it, see reasoning tokens: why you pay for more output than you see.
Why does the budget run out before the answer?
The model generates in order: first its reasoning, then the visible answer. Every token of both counts toward max_tokens. BroadNet’s docs say this for DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B: “the reasoning counts toward max_tokens”, and with a small limit “the answer comes back empty”. OpenAI’s Chat Completions reference defines max_completion_tokens the same way, as an upper bound “including visible output tokens and reasoning tokens”.
So the question to ask isn’t “how long is my answer?” It is “how long are reasoning and answer together, on this task, at this effort?” That varies more than people expect. OpenAI’s guide says its models may generate “anywhere from a few hundred to tens of thousands of reasoning tokens”, depending on the problem.
Does a bigger max_tokens cost more?
No. This is the most common reason budgets are set too tight. max_tokens is a ceiling. You are billed for the tokens actually generated. A reply that needs 1,800 tokens costs the same with a ceiling of 4,000 or 40,000.
What a tight ceiling does cost:
| Ceiling vs need | What you pay | What you get |
|---|---|---|
| Ceiling above need | Tokens generated | The answer |
| Ceiling below need | The full ceiling, all of it reasoning | Nothing usable, plus a retry |
Take an illustration. Suppose a quarter of your calls need more than 1,000 tokens. With max_tokens=1000, each of those calls bills 1,000 output tokens of thinking, returns nothing and is retried. That adds 250 wasted output tokens per call on average across all calls, before the retries themselves, plus the latency of a second request. With the ceiling at 4,000, nothing is wasted. On Qwen3.8 27B, output costs $2.17 per 1M tokens, so waste adds up at volume.
The real risk of a high ceiling is a runaway reply: a model that loops or over-thinks and generates far more than needed. A hard cap bounds that. Pick the cap from data, as below.
How do you measure what a task needs?
Run real prompts for one task type with a generous ceiling, and record the completion tokens and the reasoning share. When the model reports it, usage.completion_tokens_details.reasoning_tokens gives the reasoning part; BroadNet’s docs list that field.
import os
from openai import OpenAI
client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
MODEL = "broadnet/qwen3.8-27b"
def measure(prompts, ceiling=16_000, **kw):
"""Run real prompts with a generous ceiling and return per-call token counts."""
rows = []
for p in prompts:
r = client.chat.completions.create(model=MODEL, max_tokens=ceiling,
messages=[{"role": "user", "content": p}], **kw)
u, d = r.usage, r.usage.completion_tokens_details
reasoning = (d.reasoning_tokens or 0) if d else 0
rows.append({"reasoning": reasoning, "answer": u.completion_tokens - reasoning,
"finish": r.choices[0].finish_reason})
return rows
def pct(values, q):
values = sorted(values)
return values[min(len(values) - 1, int(q * len(values)))]
def recommend(rows, margin=1.2):
total = [r["reasoning"] + r["answer"] for r in rows]
return {"p50": pct(total, 0.5), "p95": pct(total, 0.95), "max": max(total),
"max_tokens": int(pct(total, 0.95) * margin),
"cut_off": sum(r["finish"] == "length" for r in rows)}
# print(recommend(measure(my_prompts))) # default effort
# print(recommend(measure(my_prompts, reasoning_effort="low")))
Rules for a number you can trust:
- One task type per run. Classification, extraction, code review and planning have very different distributions. One global
max_tokenseither wastes money on retries or lets runaway replies through. - 50 or more real prompts. The 95th percentile of ten samples is just the largest one.
- Run it at each effort you use.
reasoning_effort="low"changes the distribution. BroadNet documentslow; test other values before you rely on them. - If
cut_offis not zero, raise the ceiling and run again. A cut-off sample hides its true length. - Rerun when the model or prompt changes. A new model version can think more or less.
Then set max_tokens per task from recommend(). The 95th percentile plus 20% leaves most calls well under the ceiling and sends only the rare long one to the retry path below. The free reasoning token budget tool does this arithmetic in your browser once you have your numbers.
How does the context window limit the budget?
The context window holds the prompt and the answer together. BroadNet’s price feed defines context_length as “the most tokens one request can hold, prompt and answer together”. So:
max_tokens ≤ context length − prompt tokens
| Model | Context | Room for max_tokens with a 200,000-token prompt |
|---|---|---|
| GLM 5.3 | 1,048,576 | 848,576 |
| DeepSeek V4.1 Flash | 1,048,576 | 848,576 |
| Qwen3.8 27B | 262,144 | 62,144 |
This matters for long-context work. An agent that has grown its history to 250,000 tokens on Qwen3.8 27B has only about 12,000 tokens left for thinking and answer. Long-context tasks also tend to need more reasoning, so check the remaining room at the moment of the call, not once at startup. Context window and the context window calculator cover the fit.
What should you do when a reply is still cut off?
Check finish_reason on every call. If it is length, the answer is missing or incomplete, even when content has some text. Don’t parse it as if it were complete; a truncated JSON object is a common source of downstream errors (structured output).
Retry with a doubled budget, inside the room the context allows, up to a hard cap:
import json, urllib.request
FEED = json.load(urllib.request.urlopen("https://inference.broadnet.ai/pricing.json"))
CONTEXT = {m["id"]: m.get("context_length") for m in FEED["models"]}
def ask(model, messages, budget=4_000, hard_cap=32_000, **kw):
"""Retry a cut-off reply with a doubled budget, never past the cap or the context window."""
while True:
r = client.chat.completions.create(model=model, messages=messages,
max_tokens=budget, **kw)
if r.choices[0].finish_reason != "length":
return r
room = CONTEXT[model] - r.usage.prompt_tokens # context holds prompt + answer
nxt = min(budget * 2, hard_cap, room)
if nxt <= budget:
raise RuntimeError(f"still cut off at max_tokens={budget}")
print(f"cut off at {budget}, retrying with {nxt}")
budget = nxt
Two alternatives to doubling, depending on the task:
- Retry at
reasoning_effort="low"with the same budget. For simple tasks the model often over-thinks, and less effort fixes it without a bigger ceiling. - Split the task. If a reply keeps hitting a high cap, the request is usually asking for too much at once.
Log every cut-off with the task type. A rising rate means the prompt, the model or the input data changed, and the budget needs measuring again. Retries are new inferences and are billed again, so keep the attempts bounded, as in long-running agents: timeouts and retries.
How does streaming change the picture?
With stream=True, the reasoning happens before the first visible token, so a reasoning model can look stalled at the start. In a chat UI, show a “thinking” state until content arrives, and size client timeouts for the thinking time, not just the answer.
Some servers also stream the reasoning text itself, in a separate field. The name is not standard: DeepSeek’s own API uses reasoning_content, and vLLM renamed its field from reasoning_content to reasoning, warning that old client code “could silently read an empty” field. BroadNet’s docs don’t document a reasoning text field, so don’t build features on one without checking the raw response for your model. If you do read it, read it defensively:
for chunk in stream:
if chunk.choices:
delta = chunk.choices[0].delta
thinking = getattr(delta, "reasoning", None) or getattr(delta, "reasoning_content", None)
# log or display `thinking` only if your provider actually sends it
if delta.content:
print(delta.content, end="")
elif chunk.usage: # last chunk, with include_usage
print("\nreasoning tokens:", getattr(chunk.usage.completion_tokens_details, "reasoning_tokens", None))
The usage in the last chunk needs stream_options={"include_usage": True}, which BroadNet documents. More on streams in the streaming guide and SSE explained.
Which parameter name should you send?
| API | Parameter | Includes reasoning |
|---|---|---|
| OpenAI Chat Completions | max_completion_tokens (max_tokens deprecated) |
Yes |
| OpenAI Responses | max_output_tokens |
Yes |
| BroadNet Inference (Chat Completions) | max_tokens, as documented |
Yes |
| vLLM servers | max_tokens, plus optional thinking_token_budget |
Yes |
When moving code between providers, send the name the provider documents, and confirm with one test call that the limit takes effect. A limit silently ignored looks fine until a runaway reply arrives. The migration checklist covers the other parameters that change.
Where BroadNet Inference fits
BroadNet’s three reasoning models report reasoning tokens in usage when the model provides them, so the measurement above works as written. DeepSeek V4.1 Flash and GLM 5.3 give the most room, with 1,048,576 tokens of context. Qwen3.8 27B has 262,144. If you don’t need reasoning for a task, Gemma 4 VL 26B is not listed as a reasoning model and avoids the budget question altogether. On a flat DeepSeek V4.1 Flash plan, thinking tokens carry no per-token charge, but cut-off replies still cost time, so the same budgeting applies. BroadNet doesn’t document a separate reasoning cap; if you need one, use a provider or self-hosted server that offers it.
