---
title: "LLM streaming over SSE - chunks, deltas, usage, cancel"
description: "How chat-completion streaming works on the wire, how to rebuild the answer from deltas, read usage from the last chunk and cancel cleanly in Python, Node and curl."
url: https://broadrouter.com/blog/llm-streaming-sse-explained/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# LLM streaming explained: SSE chunks, deltas, usage and cancel

By the BroadNet Inference team · 4 min read · Published 2026-11-14 · Updated 2026-11-16

An LLM API streams a chat completion as Server-Sent Events: each data line holds a JSON chunk whose choices\[0\].delta carries the next piece of text. Concatenate the deltas, stop at data: \[DONE\], and set stream\_options include\_usage to get token counts in a final chunk with an empty choices list. To cancel, close the connection.

## Key takeaways

-   A stream is plain HTTP: `Content-Type: text/event-stream`, one `data:` line per chunk, a blank line between events, and `data: [DONE]` at the end.
-   Each chunk carries a `delta`, not the full message. Join `delta.content` strings to rebuild the answer.
-   Usage is off by default in streams. `stream_options: {"include_usage": true}` adds one last chunk with usage and an empty `choices` array.
-   Guard every `choices[0]` access: the usage chunk has no choices, and a parser that assumes it does will crash.
-   Cancel by closing the HTTP connection. A stream that ends without `[DONE]` is an incomplete answer, not a finished one.

**On this page**

1.  What does an SSE stream look like on the wire?
2.  What is in a chunk?
3.  How do you get token usage from a stream?
4.  Python: stream, collect usage, cancel
5.  Node: stream with an AbortController
6.  curl: see the raw events
7.  How do you cancel a stream, and what does it cost?
8.  Five streaming bugs to check for
9.  Where BroadNet Inference fits

Without [streaming](https://broadrouter.com/glossary/streaming/), a chat completion returns nothing until the whole answer is ready. A 600-token reply at about 100 tokens per second means several seconds of silence. With streaming, the first words show up as soon as the model writes them. This post covers what actually travels over the wire, how to parse it, how to get [token](https://broadrouter.com/glossary/token/) usage out of a stream, and how to stop one cleanly. For the BroadNet-specific rules in short form, see the [streaming reference guide](https://broadrouter.com/guides/streaming-chat-completions/).

## What does an SSE stream look like on the wire?

Server-Sent Events is a plain HTTP response with `Content-Type: text/event-stream`. The server keeps the connection open and writes events. Each event is one or more `data:` lines followed by a blank line. OpenAI-compatible APIs put one JSON chunk in each event and end with a literal `[DONE]`:

```text
data: {"id":"c1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}],"usage":null}

data: {"id":"c1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"One, "},"finish_reason":null}],"usage":null}

data: {"id":"c1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"two, three."},"finish_reason":"stop"}],"usage":null}

data: {"id":"c1","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":14,"completion_tokens":6,"total_tokens":20}}

data: [DONE]
```

The shape above is illustrative and trimmed. Real chunks also carry `model` and `created`, and reasoning models may report reasoning tokens in usage.

## What is in a chunk?

| Field | Where | What it means |
| --- | --- | --- |
| `choices[0].delta.role` | First chunk | Usually `assistant`. Sent once. |
| `choices[0].delta.content` | Most chunks | The next piece of text. Can be empty. |
| `choices[0].delta.tool_calls` | Tool calls | Fragments of the function name and JSON arguments, to join by `index`. |
| `choices[0].finish_reason` | Last content chunk | `stop`, `length` or `tool_calls`. `null` until then. |
| `usage` | Final chunk, if requested | Totals for the whole request. `null` everywhere else. |

The key idea is **delta, not snapshot**. Each chunk holds only what is new. You rebuild the message by joining the pieces in order. [Tool calls](https://broadrouter.com/glossary/tool-calling/) stream the same way: the arguments arrive as string fragments that are not valid JSON until the last one lands, so parse them only after `finish_reason` is `tool_calls`.

## How do you get token usage from a stream?

A non-streamed response always includes `usage`. A stream does not, unless you ask. Set `stream_options: {"include_usage": true}` and the server sends one extra chunk before `[DONE]`. That chunk has the totals and an **empty `choices` array**. On BroadNet Inference the usage can also include `prompt_tokens_details.cached_tokens` and `completion_tokens_details.reasoning_tokens` when the model reports them, which is what you need for [cost logging](https://broadrouter.com/blog/llm-api-usage-logging/).

The common bug: code that reads `chunk.choices[0]` on every chunk throws an index error on the usage chunk. Check `choices` first.

## Python: stream, collect usage, cancel

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])

stream = client.chat.completions.create(
    model="broadnet/deepseek-v4.1-flash",  # DeepSeek V4.1 Flash, a reasoning model
    messages=[{"role": "user", "content": "Explain SSE in three sentences."}],
    stream=True,
    stream_options={"include_usage": True},
    max_tokens=2000,  # reasoning counts toward this limit
)

def user_pressed_stop():  # replace with your own condition
    return False

parts, usage, finish = [], None, None
try:
    for chunk in stream:
        if chunk.choices:
            choice = chunk.choices[0]
            if choice.delta.content:
                parts.append(choice.delta.content)
                print(choice.delta.content, end="", flush=True)
            if choice.finish_reason:
                finish = choice.finish_reason
        if chunk.usage:
            usage = chunk.usage
        if user_pressed_stop():  # your own condition
            break
finally:
    stream.close()  # closes the HTTP connection

text = "".join(parts)
```

`break` alone stops reading, but the socket stays open until it is garbage collected. Close it explicitly so the server sees the disconnect.

## Node: stream with an AbortController

```javascript
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://inference.broadnet.ai/v1", apiKey: process.env.BROADNET_API_KEY });
const controller = new AbortController();
setTimeout(() => controller.abort(), 30_000); // hard stop after 30 s

const stream = await client.chat.completions.create(
  {
    model: "broadnet/deepseek-v4.1-flash",
    messages: [{ role: "user", content: "Explain SSE in three sentences." }],
    stream: true,
    stream_options: { include_usage: true },
  },
  { signal: controller.signal },
);

let text = "";
for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) text += delta;
  if (chunk.usage) console.log(chunk.usage);
}
```

The [openai-node README](https://github.com/openai/openai-node) documents the `signal` option, and it works while the body is still being read. In a web server, wire `controller.abort()` to the client request’s `close` event so a user who leaves the page stops the generation too. More setup in the [Node SDK guide](https://broadrouter.com/guides/openai-node-sdk/).

## curl: see the raw events

```bash
curl -N https://inference.broadnet.ai/v1/chat/completions \
  -H "Authorization: Bearer $BROADNET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"broadnet/gemma-4-26b-a4b","stream":true,"stream_options":{"include_usage":true},"messages":[{"role":"user","content":"Count to five."}]}'
```

`-N` turns off output buffering so events print as they arrive. Press Ctrl+C to cancel; that closes the connection. The [API request builder](https://broadrouter.com/tools/api-request-builder/) generates this request, and the Python and Node versions, for any model and base URL.

## How do you cancel a stream, and what does it cost?

Cancellation in SSE is just closing the connection. There is no “stop” message. What happens next is up to the server: a well-behaved one notices the closed socket and stops generating. Tokens produced before that moment were real work. BroadNet’s API reference does not document billing for cancelled streams, so plan as if generated tokens up to the disconnect are billed on [pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/), and check your usage records. On a flat plan, a cancelled stream costs nothing extra but frees the session only when the server stops.

## Five streaming bugs to check for

1.  **Buffering proxies.** Nginx and some CDNs buffer responses, so the client gets everything at once. Disable proxy buffering for the streaming route.
2.  **No `[DONE]` check.** A dropped connection looks like a short answer. Treat a missing `[DONE]` as incomplete and record the `X-Request-ID`, as the [errors guide](https://broadrouter.com/guides/errors-and-retries/) describes.
3.  **Retrying mid-stream.** SDKs do not retry once the body starts arriving. A retry is a new request and, on pay-as-you-go, a new bill.
4.  **Ignoring `finish_reason: length`.** On reasoning models such as [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) the thinking counts toward [max\_tokens](https://broadrouter.com/glossary/max-tokens/), so a small limit can stop the stream before any visible text. [What reasoning tokens cost](https://broadrouter.com/blog/reasoning-tokens-cost/).
5.  **Splitting lines wrong.** If you parse SSE by hand, buffer bytes until you see a blank line. A TCP read can end in the middle of a JSON chunk.

## Where BroadNet Inference fits

BroadNet Inference uses the standard [OpenAI-compatible](https://broadrouter.com/glossary/openai-compatible-api/) streaming shape, so the Python and Node SDKs parse it unchanged. Usage in the final chunk includes cached and reasoning tokens when the model reports them. Its docs say streaming should still be verified per model before production use, so run the code above against the exact model you ship. If you mostly care about how fast the first token arrives, read [TTFT vs throughput](https://broadrouter.com/blog/llm-latency-ttft-throughput/) next. If you need realtime voice over WebSockets, this API is not built for that; it offers chat completions and file transcription.

Free tools for this

-   [OpenAI to BroadNet migration helper](https://broadrouter.com/tools/openai-to-broadnet-migration/) Paste OpenAI SDK code; get base URL, key and model edits, a diff and the checks that remain.
-   [max\_tokens calculator for reasoning models](https://broadrouter.com/tools/reasoning-token-budget/) Answer length + reasoning budget + margin = a max\_tokens that does not cut the answer off.
-   [JSON schema generator for LLM structured output](https://broadrouter.com/tools/json-schema-generator/) Sample JSON to strict schema, Pydantic, Zod and a ready request.
-   [LLM JSON output validator](https://broadrouter.com/tools/llm-json-output-validator/) Parse errors by line and column, safe repairs, cut-off detection and JSON Schema checks for model output.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Streaming](https://broadrouter.com/glossary/streaming/)

Streaming is a way of returning a language model response in small chunks while it is still being generated, instead of as one reply at the end. OpenAI-compatible APIs stream over server-sent events when the request sets stream to true.

[Server-sent events (SSE)](https://broadrouter.com/glossary/sse/)

Server-sent events (SSE) is a web standard for a server to push a stream of text events to a client over one long-lived HTTP response with the content type text/event-stream. Each event is one or more field lines, such as data:, ended by a blank line.

[Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/)

Pay-as-you-go is a pricing model in which you pay only for what you use, metered per unit such as tokens, requests or audio minutes, with no subscription or minimum commitment. Most LLM APIs bill this way, from a prepaid balance or a periodic invoice.

[Reasoning tokens](https://broadrouter.com/glossary/reasoning-tokens/)

Reasoning tokens are tokens a reasoning model generates to work through a problem before it writes its visible answer. They are usually billed as output tokens, count toward the output limit and the context window, and may be hidden or summarized by the API.

[Base URL](https://broadrouter.com/glossary/base-url/)

A base URL is the fixed root address of an API, such as https://api.example.com/v1, that a client puts in front of every endpoint path, such as /chat/completions. In OpenAI-compatible SDKs, changing the base URL, together with the API key and model ID, points the same code at a different provider.

[Chat Completions API](https://broadrouter.com/glossary/chat-completions-api/)

The Chat Completions API is OpenAI's HTTP endpoint, POST /v1/chat/completions, that takes a model name and a list of role-tagged messages and returns the model's next message. Its request and response shape became the common format that OpenAI-compatible providers and SDKs reuse.

[max\_tokens](https://broadrouter.com/glossary/max-tokens/)

max\_tokens is an LLM API parameter that sets the most tokens a model may generate in one response. When generation reaches the limit, it stops and the response is marked as cut off, for example with finish\_reason set to "length".

## Questions

### What is the difference between SSE and WebSockets for LLM streaming?

SSE is one-way, server to client, over a normal HTTP response. That is all a chat completion needs, so OpenAI-compatible APIs use SSE. WebSockets are two-way and need a protocol upgrade; they are used for realtime voice, not for chat completions.

### Why is usage null in my streaming chunks?

Usage is only sent when you ask for it. Add stream\_options with include\_usage set to true. Every chunk except the last then has usage null, and the last chunk carries the totals with an empty choices list.

### Am I billed if I cancel a stream?

Providers differ and many do not document it. Assume tokens generated before the server noticed the closed connection are billed. Log the X-Request-ID and compare against your usage records.

### How do I know a stream finished normally?

You received data: \[DONE\] and the last content chunk had a finish\_reason such as stop. A finish\_reason of length means max\_tokens cut the answer. No \[DONE\] means the connection dropped.

### Does streaming make the model faster?

No. Total generation time is the same. Streaming lowers the time until the user sees the first words, which is what makes a chat interface feel fast.

## Keep reading

-   [LLM latency: TTFT vs tokens per second vs end-to-end time](https://broadrouter.com/blog/llm-latency-ttft-throughput/)
-   [LLM API usage logging: cost per user and feature, PII redacted](https://broadrouter.com/blog/llm-api-usage-logging/)
-   [How to handle LLM API 429 errors: backoff, jitter and limiters](https://broadrouter.com/blog/handle-llm-api-429-rate-limits/)
-   [A private endpoint starts with a clear workload](https://broadrouter.com/blog/private-endpoint-planning/)
-   [Before fine-tuning: define the problem to solve](https://broadrouter.com/blog/fine-tuning-readiness/)
-   [Guide: Stream chat completions and handle dropped connections](https://broadrouter.com/guides/streaming-chat-completions/)
-   [Guide: Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)
-   [Guide: Use the OpenAI Node.js SDK with BroadNet Inference](https://broadrouter.com/guides/openai-node-sdk/)
-   [Guide: BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-llm-streaming-sse-explained&utm_content=page-cta) ·

**Sources**

-   [BroadNet Inference API reference, streaming and usage](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-llm-streaming-sse-explained&utm_content=sources#requests)
-   [MDN, Using server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events)
-   [OpenAI community announcement, usage stats in streaming (stream\_options)](https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156)
-   [openai-python README, streaming responses and retries (fetched 2026-11-14)](https://github.com/openai/openai-python)
-   [openai-node README, AbortSignal cancellation (fetched 2026-11-14)](https://github.com/openai/openai-node)
