Most of what an agent sends is the same on every turn. Prompt caching lets the provider skip recomputing that repeated part, and bill it for less.
How it works
When a model reads a prompt, it computes internal state for every token. A provider can keep that state for a short time. If the next request starts with the same tokens, the provider reads the stored state instead of recomputing it. Those tokens are cached input tokens.
Caches match from the start of the prompt. OpenAI’s guide says reuse “requires the entire rendered prefix to match.” One changed character early in the prompt makes everything after it new input.
Providers differ in the details:
- OpenAI enables caching by default on supported models and reports hits in the usage block. It documents a 1,024-token minimum prefix on its newer models.
- Anthropic bills cache reads at 0.1 times the base input price for most models, bills cache writes at a premium, and keeps entries for 5 minutes by default.
In the Chat Completions format, hits appear as usage.prompt_tokens_details.cached_tokens. Cache rate is that number divided by prompt_tokens.
Why it matters
For long, repeated prompts, the cache rate decides the bill. An agent that resends a 200,000-token repository on every turn pays the full input price once and the cached price after that, as long as the prefix stays stable. Caching can also shorten time to first token on long prompts, because less work is done before generation starts.
To raise the rate, put stable content first (system prompt, tool definitions, documents) and changing content last. Keep timestamps and request IDs out of the top of the prompt, and keep tool lists in a fixed order. What share of providers’ published prices include a cached-input price, and the typical discount, is in the LLM API price index.
On BroadNet Inference
BroadNet publishes cached input prices for three chat models, per 1M tokens:
| Model | New input | Cached input |
|---|---|---|
| DeepSeek V4.1 Flash | $0.175 | $0.0055 |
| GLM 5.3 | $0.78 | $0.173 |
| Qwen3.8 27B | $0.175 | $0.0435 |
Cached tokens are billed at the lower price automatically, and prompt_tokens_details.cached_tokens shows them when the model reports them. In a stream, request usage with stream_options as shown in the streaming guide.
On a flat-rate plan there is no token meter, so the cache rate does not change the bill. The full walkthrough, with code to log your rate, is in cached input tokens explained.