---
title: "Where AI agent tokens go, and how to cut them | BroadRouter"
description: "System prompt, tool schemas, file reads, history and reasoning: how an agent loop spends tokens, how to measure each part from usage data, and what to trim first."
url: https://broadrouter.com/blog/agent-token-budget/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Agent token budget: where an agent loop's tokens go

By the BroadNet Inference team · 5 min read · Published 2026-11-14

An agent loop resends its whole context on every call: system prompt, tool definitions, conversation history, file contents and tool output, plus the new step. Input usually outweighs output many times over. Log prompt\_tokens, cached\_tokens and completion\_tokens per call, find the biggest repeating block, then trim tools, cap tool output and restart long sessions.

## Key takeaways

-   Every model call resends the full context. A 40-step task with 30,000 tokens per call reads 1.2 million input tokens.
-   The five buckets: system prompt, tool schemas, history, file reads and tool output, and output including reasoning.
-   Measure per call from the usage block: prompt\_tokens, prompt\_tokens\_details.cached\_tokens and completion\_tokens.
-   Cursor reported that its system prompt and tool definitions had become one of the largest sources of spend it fully controls.
-   Trim in this order: unused tools, oversized tool output, stale history, then the system prompt. Keep the stable prefix identical so it stays cached.

**On this page**

1.  Why does an agent resend everything?
2.  Where do the tokens go?
3.  How do you measure it?
4.  What should you trim first?
5.  Why keep the prefix stable?
6.  What does a worked agent budget look like?
7.  Where BroadNet Inference fits

A chat costs what you type plus what the model writes. An agent costs far more, because it calls the model dozens of times and resends almost everything each time. If you want to cut that cost, or just stop hitting the [context window](https://broadrouter.com/glossary/context-window/), you need to know which part of each request is big. This post breaks an agent loop into five buckets, shows how to measure each from the API’s own usage data, and lists what to trim first.

## Why does an agent resend everything?

Chat-completions APIs are stateless. The model remembers nothing between calls, so the agent sends the full context every step. A 40-step task where each call carries 30,000 tokens reads **1.2 million input tokens** to produce maybe 40,000 output tokens. That 30-to-1 ratio is why [input tokens](https://broadrouter.com/glossary/input-and-output-tokens/) and their cache price dominate agent bills.

## Where do the tokens go?

| Bucket | What it is | Repeats every call? | Typical driver |
| --- | --- | --- | --- |
| System prompt | Agent instructions, rules files, project notes | Yes | Long rules files |
| Tool schemas | Name, description and JSON Schema of every tool | Yes | Built-in tools plus every MCP server |
| History | Earlier messages, tool calls and results | Yes, and grows | Long sessions |
| File reads and tool output | File contents, test logs, search results | Yes, until dropped | Reading whole files, verbose commands |
| Output | Visible answer, tool calls and reasoning | No | Reasoning effort, rewriting whole files |

The first two are fixed overhead. Cursor wrote in September 2026 that the system prompt and tool definitions “had become one of the largest sources of spend that we fully control”, and called file reads “another large source of token spend”. Its changes, including cutting about two-thirds of the system prompt and loading tools used in fewer than 20% of conversations only when needed, cut users’ token costs by 7%. GitHub’s VS Code team named the same two costs for Copilot: the repeated prompt prefix and tool-definition overhead.

## How do you measure it?

You don’t need a tracing product. The usage block on each response is enough.

1.  **Turn on usage.** For streaming, add `stream_options: {"include_usage": true}`; usage arrives in the last chunk. See [streaming](https://broadrouter.com/guides/streaming-chat-completions/).
2.  **Log per call:** step number, `prompt_tokens`, `prompt_tokens_details.cached_tokens`, `completion_tokens` and, when reported, `completion_tokens_details.reasoning_tokens`.
3.  **Measure the fixed overhead once.** Send one request with only the system prompt and tools, and the word “hi”. Its `prompt_tokens` is your floor.
4.  **Measure a tool’s output.** Count characters of each tool result before you append it. Dividing by four gives a rough English token count; paste a sample into the free [token counter](https://broadrouter.com/tools/token-counter/) for the exact figure on your model’s tokenizer.

```python
r = client.chat.completions.create(model=MODEL, messages=msgs, tools=tools)
u = r.usage
cached = getattr(u.prompt_tokens_details, "cached_tokens", 0) or 0
log.info("step=%d prompt=%d cached=%d out=%d", step, u.prompt_tokens, cached, u.completion_tokens)
```

Plot `prompt_tokens` by step. A steady slope means history is growing. A sudden jump means a large file or log entered the context. A high floor means too many tools.

In Aider, `/tokens` reports what’s in the current context, and `/drop` removes files from it.

## What should you trim first?

Order by size times repetitions, not by what’s easiest:

1.  **Unused tools.** Every tool schema rides on every call. OpenAI’s guide says function definitions “are billed as input tokens” and suggests fewer than 20 functions at the start of a turn, as a soft rule. Disable MCP servers you don’t use in this task; opencode’s docs say plainly that MCP tools consume model context.
2.  **Tool output.** Cap command output (last 200 lines of a test log, not 20,000). Return search results as paths and line numbers, then read only what’s needed.
3.  **File reads.** Read the function, not the file. Aider’s repo map uses a default 1k-token budget (`--map-tokens`) instead of whole files.
4.  **History.** Start a new task per change. When a session must continue, replace old turns with a short summary.
5.  **System prompt.** Remove rules the model already follows. Keep the rest stable.
6.  **Output.** Ask for diffs, not whole files. Set `reasoning_effort` to `low` for routine steps. [Reasoning tokens](https://broadrouter.com/glossary/reasoning-tokens/) are billed as output.

## Why keep the prefix stable?

Providers cache a repeated prefix and bill it at a lower rate. On BroadNet Inference, [DeepSeek V4.1 Flash](https://broadrouter.com/models/deepseek-v4.1-flash/) bills [cached input](https://broadrouter.com/glossary/cached-input-tokens/) at $0.0055 per 1M tokens and new input at $0.175. [GLM 5.3](https://broadrouter.com/models/glm-5.3/) bills $0.173 against $0.78. A timestamp in the system prompt, or tools listed in a different order each call, breaks the cache from that point on. Put stable parts first, in a fixed order, and changing parts last. More in [cached input tokens explained](https://broadrouter.com/blog/cached-input-tokens-explained/); the [prompt caching calculator](https://broadrouter.com/tools/prompt-caching-calculator/) shows what a given hit rate saves.

Caching lowers the price of the prefix, but not its size. Cached tokens still fill the context window and still take time to process.

## What does a worked agent budget look like?

Take the 40-step task above, with 80% of input cached:

| Part | Tokens | Billed at |
| --- | --- | --- |
| New input | 240,000 | Input price |
| Cached input | 960,000 | Cached input price |
| Output incl. reasoning | 40,000 | Output price |

Cutting 5,000 tokens of unused tool schemas from each call removes 200,000 input tokens from this task, about a sixth of the total, with no change to the agent’s behaviour. For the cost levers beyond trimming, such as model choice, caching and flat plans, see [10 ways to cut coding agent LLM costs](https://broadrouter.com/blog/cut-coding-agent-llm-costs/). For multi-agent setups, multiply by the number of agents; see [what multi-agent setups cost](https://broadrouter.com/blog/multi-agent-cost/). To model your own loop step by step, use the [agent run cost estimator](https://broadrouter.com/tools/agent-run-cost-estimator/).

## Where BroadNet Inference fits

BroadNet Inference returns prompt and completion tokens in the usage block, plus cached and reasoning tokens when the model reports them, so the measurement above works as written. Its low cached-input price on DeepSeek V4.1 Flash rewards a stable prefix. If trimming isn’t worth your time, the unlimited plans remove the token meter for DeepSeek V4.1 Flash at $60 a month per session; the context window still applies, and a lean context still runs faster. Plans are on [inference.broadnet.ai/plans](https://inference.broadnet.ai/plans?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-agent-token-budget&utm_content=body).

Free tools for this

-   [Agent run cost estimator](https://broadrouter.com/tools/agent-run-cost-estimator/) Context growth per step, total tokens and cost of one agent run, with a DeepSeek session comparison.
-   [LLM batch job planner](https://broadrouter.com/tools/llm-batch-job-planner/) Run time, binding limit, cost per model and the flat sessions needed to finish a big job by a deadline.
-   [Flat-rate vs per-token LLM cost calculator](https://broadrouter.com/tools/flat-rate-vs-per-token-calculator/) Break-even between an unlimited plan and paying per token, per agent.
-   [LLM rate limit and concurrency calculator](https://broadrouter.com/tools/llm-rate-limit-calculator/) Which limit binds first: RPM, TPM or concurrency, with Little's law.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

[Input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/)

Input tokens are the tokens a language model reads in a request, including the system prompt, conversation history, tool definitions and images. Output tokens are the tokens it generates in reply, including any reasoning. LLM APIs usually price them separately, and output usually costs more.

[System prompt](https://broadrouter.com/glossary/system-prompt/)

A system prompt is the instruction text an application sends to a language model ahead of the conversation, usually as a message with the system or developer role. It sets the model's role, rules, tone and output format, and the model is meant to follow it over conflicting user messages.

[Cached input tokens](https://broadrouter.com/glossary/cached-input-tokens/)

Cached input tokens are prompt tokens that an LLM provider has recently processed and can reuse from a cache instead of computing again, usually a repeated prefix such as a system prompt or document. Providers bill them at a lower rate than new input tokens.

[Context window](https://broadrouter.com/glossary/context-window/)

A context window is the maximum number of tokens a language model can work with in a single request. It covers the prompt, conversation history, tool definitions, images and the generated output, including any reasoning tokens. Anything that does not fit must be cut, summarised or split across several requests.

[MCP (Model Context Protocol)](https://broadrouter.com/glossary/mcp/)

MCP (Model Context Protocol) is an open protocol, built on JSON-RPC 2.0, that standardises how LLM applications connect to external tools and data. An MCP server exposes tools, resources and prompts; an MCP client inside the host application discovers them and passes tool calls from the model to the server.

[Reasoning tokens](https://broadrouter.com/glossary/reasoning-tokens/)

Reasoning tokens are tokens a reasoning model generates to work through a problem before it writes its visible answer. They are usually billed as output tokens, count toward the output limit and the context window, and may be hidden or summarized by the API.

## Questions

### Why does my coding agent use so many input tokens?

Because the API is stateless. Each call resends the system prompt, every tool definition, the conversation so far and all file contents still in context. A long session multiplies that by the number of steps.

### How do I see how many tokens each agent step uses?

Read the usage block of each response. With streaming, set stream\_options include\_usage to true and read the last chunk. Log prompt\_tokens, prompt\_tokens\_details.cached\_tokens and completion\_tokens with a step number.

### Do tool definitions count as input tokens?

Yes. The tools list is part of the prompt and is sent on every call. MCP servers add their tools to that list, which is why opencode's docs say to add only the servers you need.

### Does prompt caching fix the problem?

It cuts the price of the repeated prefix, not its size. Cached input still fills the context window. On BroadNet, DeepSeek V4.1 Flash bills cached input at $0.0055 per 1M tokens against $0.175 for new input.

### Are reasoning tokens part of the budget?

Yes, on the output side. Reasoning models count thinking toward max\_tokens and bill it as output on pay-as-you-go. Set reasoning\_effort to low for routine steps.

## Keep reading

-   [OpenClaw and Hermes Agent on a cheap model: setup and token math](https://broadrouter.com/blog/openclaw-hermes-agent-cheap-model/)
-   [Cached input tokens: how to cut an LLM API bill](https://broadrouter.com/blog/cached-input-tokens-explained/)
-   [What multi-agent setups really cost: per token vs flat sessions](https://broadrouter.com/blog/multi-agent-cost/)
-   [Best open model for coding agents: pick by tools, context and cost](https://broadrouter.com/blog/best-open-model-for-coding-agents/)
-   [Flat-rate vs pay-per-token LLM API: the break-even math](https://broadrouter.com/blog/flat-rate-vs-pay-per-token-llm-api/)
-   [Reduce prompt tokens: measured cuts for system prompts, tools, data](https://broadrouter.com/blog/reduce-prompt-tokens/)
-   [Claude Code with open models: proxies, native endpoints and limits](https://broadrouter.com/blog/claude-code-with-openai-compatible-models/)
-   [Prompt injection in coding agents: defences that actually hold](https://broadrouter.com/blog/prompt-injection-coding-agents/)
-   [Guide: Stream chat completions and handle dropped connections](https://broadrouter.com/guides/streaming-chat-completions/)
-   [Guide: Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)
-   [Guide: Run Aider on unlimited DeepSeek V4.1 Flash](https://broadrouter.com/guides/aider-unlimited-deepseek/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-agent-token-budget&utm_content=page-cta) ·

**Sources**

-   [Cursor, improved token efficiency (September 23, 2026; fetched 2026-11-14)](https://cursor.com/blog/improved-token-efficiency)
-   [VS Code blog, improving token efficiency in GitHub Copilot (fetched 2026-11-14)](https://code.visualstudio.com/blogs/2026/06/17/improving-token-efficiency-in-github-copilot)
-   [OpenAI, function calling guide, fewer than 20 functions (fetched 2026-11-14)](https://developers.openai.com/api/docs/guides/function-calling)
-   [opencode docs, MCP servers and context use (fetched 2026-11-14)](https://opencode.ai/docs/mcp-servers/)
-   [Aider docs, repository map and in-chat commands (fetched 2026-11-14)](https://aider.chat/docs/repomap.html)
-   [BroadNet Inference API reference, usage and cached tokens (fetched 2026-11-14)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-agent-token-budget&utm_content=sources)
