Speed has two parts: how long before anything appears, and how fast text flows after that. Tokens per second measures the second part.
How it works
NVIDIA’s benchmarking docs define the related metrics:
- Time to first token (TTFT): from sending the request to receiving the first token. It grows with prompt length.
- Inter-token latency (ITL): the average gap between output tokens.
- Per-user TPS: one request’s output tokens divided by its end-to-end time, so in NVIDIA’s definition it includes TTFT. Many dashboards instead quote decode speed after the first token, as this page does.
- System TPS: total output tokens across all simultaneous requests, also called throughput.
They pull in opposite directions. NVIDIA notes that as concurrency rises, system TPS goes up while per-user TPS goes down. A provider’s marketing number and your single request’s speed can both be honest and still differ.
A quick estimate for one reply:
total time ≈ TTFT + output tokens ÷ tokens per second
Take an illustrative stream of 50 tokens a second: a 1,000-token answer takes about 20 seconds after the first token. If the model also writes 2,000 reasoning tokens, add about 40 seconds before the answer starts. The time to first token comes on top.
Why it matters
- User experience. Streaming shows text as it arrives, so TTFT and TPS decide how fast a reply feels.
- Agent throughput. A coding agent waits for each reply before its next step. Faster output means more steps per hour.
- Plan limits. On a plan with no token meter, speed becomes the real ceiling on how many tokens you can use.
On BroadNet Inference
BroadNet’s plans page states a typical speed of about 100 tokens per second for DeepSeek V4.1 Flash. A session on the unlimited plans runs one request at a time, so at that speed a session that generated non-stop for 30 days would produce about 259 million output tokens. Real agents spend time reading, running tools and waiting, so the practical figure is lower.
To get more total throughput on a plan, add sessions: one per agent working in parallel. See how unlimited sessions work and running parallel coding agents.
Common mistakes
- Comparing providers on a single request. Measure TTFT and TPS over many requests, with your own prompt sizes.
- Forgetting reasoning time when setting client timeouts.