BroadRouter

Reduce prompt tokens: measured cuts for system prompts, tools, data

By the BroadNet Inference team · 7 min read · Published · Updated

Measure first: log prompt_tokens per call, find the biggest part, and diff with the model's tokenizer. Across five tokenizers, trimming a wordy system prompt saved 68 to 71% and CSV used 31 to 41% fewer tokens than minified JSON. On three models, a trimmed prompt with 3 short tools instead of 10 verbose ones cut the full request by about 65 to 71%.

Shorter prompts cost less, leave more of the context window for work, and usually return the first token sooner. But “make the prompt shorter” is vague advice. This post measures common cuts on one realistic support-bot prompt, with the real tokenizers of four open models and OpenAI’s o200k_base, so you can see which edits are worth your time.

If you run a coding agent, the agent token budget covers that loop. This post is about application prompts: chatbots, classifiers and extraction jobs.

How did we measure?

All counts below are ours, made on 15 November 2026:

  • Tokenizers. The official tokenizer.json files for DeepSeek V4.1 Flash, GLM 5.3, Qwen3.8 27B and Google’s Gemma 4 26B A4B, each at a pinned Hugging Face revision (see Sources), loaded with Hugging Face tokenizers 0.23.3. Plus o200k_base from tiktoken 0.14.0, for comparison with GPT models. Special tokens were not added.
  • Rendered prompts. For DeepSeek, GLM and Qwen, we also rendered full requests through each model’s published chat_template.jinja (Jinja2 sandbox), with system prompt, tools, history and question, and counted the result. That is close to what a server tokenises, but a provider can use a different template, so treat the API’s usage.prompt_tokens as the final word.
  • Text. A synthetic support bot for a fictional telecom: a 261-word system prompt, 10 tools, 8 few-shot examples, 40 order rows, a 20-turn history and a 31-word Arabic paragraph. One prompt is one data point. Your savings will differ; the method carries over.

BroadNet’s Gemma 4 VL 26B is based on Google’s model, so its count is an estimate. The others use the models’ own files.

Which cuts save the most tokens?

Raw text, before → after, with the share saved:

Change DeepSeek V4.1 Flash GLM 5.3 Qwen3.8 27B Gemma 4 GPT o200k
System prompt: 261 words → 66 words 295 → 88 (70%) 294 → 89 (70%) 297 → 95 (68%) 297 → 96 (68%) 293 → 86 (71%)
Few-shot: 8 examples → 3 157 → 69 (56%) 157 → 69 (56%) 156 → 71 (54%) 156 → 71 (54%) 152 → 69 (55%)
40 rows: indented JSON → minified JSON 2,446 → 1,606 (34%) 2,460 → 1,581 (36%) 3,180 → 1,980 (38%) 3,210 → 2,010 (37%) 2,446 → 1,567 (36%)
40 rows: minified JSON → CSV 1,606 → 948 (41%) 1,581 → 960 (39%) 1,980 → 1,359 (31%) 2,010 → 1,382 (31%) 1,567 → 946 (40%)
10 tools: verbose → short descriptions 1,224 → 806 (34%) 1,214 → 797 (34%) 1,225 → 806 (34%) 1,252 → 827 (34%) 1,202 → 796 (34%)
10 verbose tools → 3 short ones 1,224 → 258 (79%) 1,214 → 257 (79%) 1,225 → 266 (78%) 1,252 → 276 (78%) 1,202 → 255 (79%)
Arabic paragraph: fully vowelled → plain 193 → 74 (62%) 267 → 78 (71%) 151 → 62 (59%) 180 → 64 (64%) 218 → 62 (72%)

The percentages are similar across tokenizers for English prose, so a cut that helps on one model helps on the others. Two rows break that pattern. The order data cost Qwen and Gemma 23 to 46% more tokens than DeepSeek and GLM in every format, because their tokenizers split numbers into single digits: 100200 is 6 tokens on Qwen and 2 on DeepSeek. And Arabic diacritics hit some tokenizers far harder than others.

What does a whole request look like?

Raw text misses the chat template. Rendered through each model’s own template, with the same customer question, the full prompt measured:

Request DeepSeek V4.1 Flash GLM 5.3 Qwen3.8 27B
A. Verbose system prompt + 10 verbose tools 1,677 1,535 1,797
B. Trimmed system prompt + 10 short tools 1,052 913 1,176
C. Trimmed system prompt + 3 short tools 574 443 636
D. Trimmed system prompt, no tools 136 121 174
C + 20 earlier turns in full 2,374 2,243 2,693
C + 3-line summary + last 4 turns 979 849 1,103

Going from A to C cut about 65 to 71% of every request (64.6% on Qwen, 71.1% on GLM). Most of that came from the tools. Rendered with the verbose system prompt, the 10 verbose tools added 1,334 tokens on DeepSeek, 1,209 on GLM and 1,421 on Qwen; the 3 short tools in C added 322 to 462 over D. Tool calling schemas ride on every request, whether the model uses them or not.

At volume this adds up. On Qwen3.8 27B, A to C saves 1,161 tokens a request. At 100,000 requests a month that is about 116 million input tokens, or 116 × $0.175 a month at the standard input price.

How do you trim a system prompt without breaking it?

Our 261-word prompt and 66-word prompt carry the same seven rules. The cuts were all of the same kind:

  1. Drop politeness to the model. “It is very important that you always remain polite and courteous at all times” became nothing: the role line implies it.
  2. One rule per line. Bullets replace “Please make sure that you always…” sentences.
  3. Merge duplicates. Three sentences about language became “Reply in the customer’s language (Arabic or English), briefly.”
  4. Delete restated defaults. “Remember to always be helpful!” changes nothing.

Then test. A shorter prompt that loses a rule costs more than it saves. Run both versions over the same 50 to 100 real inputs and compare, as in evaluating an LLM before switching.

How many few-shot examples do you need?

Fewer than most prompts carry. Each few-shot example is paid on every call. Keep the ones that fix a mistake the model makes without them, such as a dialect example for an Arabic classifier, and drop the ones it already gets right. Our 8 → 3 cut saved 54 to 56% of the examples block. Whether 3 is enough for your task is an accuracy question, not a token question, so check it on your test set.

Should you send tools, data and history differently?

Tools. Send only the tools the current step can use. A billing question needs get_account, get_bill and create_ticket, not order_esim. Shorten descriptions to what the model needs to choose and fill the tool. Indentation doesn’t matter here: the templates we rendered all re-serialise the tools array with tojson, so a pretty-printed tools array costs the same as a minified one.

Data in messages. Data you paste into a message is sent as written. Minify JSON at least. For flat records, CSV or TSV drops the repeated keys: 31 to 41% fewer tokens than minified JSON in our test. Nested data stays JSON. Check the model still reads the columns correctly.

History. Every past turn is resent. Replacing 16 old turns with a three-line summary, and keeping the last four turns verbatim, cut the Qwen prompt from 2,693 to 1,103 tokens. A summary drops detail the user may refer back to, and rewriting it changes the prefix, which breaks caching. Long-running agent context management covers when to compact.

Arabic diacritics. Most Arabic is written without short vowels. If your input is fully vowelled, from a dictionary, textbook or religious source, stripping the marks (U+064B to U+0652) cut our paragraph by 59 to 72%. Keep them when the task depends on them, such as teaching or poetry. Arabic text normalization covers the other characters worth normalising, and tokens per word in Arabic the wider gap with English.

Where does caching fit?

Caching doesn’t shrink the prompt. It makes the repeated part cheaper. OpenAI’s caching guide says “the entire rendered prefix” must match and advises putting stable instructions first. BroadNet’s pricing page describes cached input as “usually the repeated start of recent requests, such as a long system prompt”, with nothing to set up, and its docs report hits in usage.prompt_tokens_details.cached_tokens.

So the order is: shorten first, then make the shortened prefix stable. System prompt and tools first, byte-identical on every call; history next, append-only; the new message last. On Qwen3.8 27B, cached input is $0.0435 per 1M against $0.175. Prompt structure for caching has the layout and a test that proves the prefix is reused. Cached tokens still count toward the context window.

How do you measure your own prompt?

Start with the bill. Log usage.prompt_tokens per call site for a day and find the largest. Then compare versions locally with this script, which prints before and after counts for two files:

# pip install tokenizers tiktoken huggingface_hub
import json, sys
import tiktoken
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer

def hf(repo, rev):
    return Tokenizer.from_file(hf_hub_download(repo, "tokenizer.json", revision=rev))

COUNTERS = {
    "Qwen3.8 27B": hf("Qwen/Qwen3.8-27B", "37561816e66b5b9153b1b64c10968c18c1e6f4a0").encode,
    "GPT o200k": tiktoken.get_encoding("o200k_base").encode,
}

def n(enc, text):
    out = enc(text)
    return len(out.ids if hasattr(out, "ids") else out)

def diff(before, after):
    for name, enc in COUNTERS.items():
        a, b = n(enc, before), n(enc, after)
        print(f"{name:12} {a:6} -> {b:6}  saved {a - b} ({100 * (a - b) / a:.0f}%)")

if __name__ == "__main__":
    diff(open(sys.argv[1], encoding="utf-8").read(), open(sys.argv[2], encoding="utf-8").read())

Running it on our two system prompts printed:

Qwen3.8 27B     297 ->     95  saved 202 (68%)
GPT o200k       293 ->     86  saved 207 (71%)

Add the other models’ repos from the Sources list to compare all four. For a quick check in the browser, the prompt token diff compares two versions side by side, and the token counter counts one. After you deploy, confirm the saving in usage.prompt_tokens, which includes the template.

Where BroadNet Inference fits

BroadRouter is BroadNet’s site. BroadNet Inference bills prompt tokens at each model’s input price and reads repeated prefixes from cache at the cached price where the model has one, so both shorter prompts and stable prefixes lower the bill. Gemma 4 VL 26B has a 32,768-token context, where trimming matters for fit as well as cost. On DeepSeek V4.1 Flash plans there is no token meter, but the 1,048,576-token context still applies, and a shorter prompt still returns sooner. Prompts are not retained after each request.

Questions

How do I reduce the number of tokens in my prompt?

Measure first: log usage.prompt_tokens and find the largest part. Then, in order of what we measured saving most: send only the tools a step needs, shorten tool and system prompt wording, send tables as CSV not JSON, cut few-shot examples to the ones that change results, and summarise old chat turns.

Does pretty-printed JSON use more tokens?

Yes. In our test, minifying 40 rows of indented JSON saved 34 to 38% of the tokens, depending on the tokenizer, and sending them as CSV saved 57 to 61%. Put the other way, indented JSON took 52 to 61% more tokens than minified JSON and 132 to 159% more than CSV. Tool schemas are an exception: chat templates re-serialise them, so indentation in your tools array does not reach the model.

How many tokens do tool definitions use?

It depends on wording and the model's template. Our 10 tools with verbose descriptions added about 1,200 to 1,400 tokens to a rendered prompt on DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B. Three tools with short descriptions added about 320 to 460.

Do Arabic diacritics increase token count?

A lot. A 31-word MSA paragraph took 62 to 78 tokens without diacritics and 151 to 267 tokens fully vowelled, across five tokenizers. Most Arabic text is written without them, so strip them from input unless the task, such as poetry or teaching, needs them.

Is prompt caching better than shortening the prompt?

They work together. Caching makes a repeated prefix cheaper, but it still counts toward the context window and still has a price. Shorten first, then keep the shortened prefix byte-identical at the start of every request so it can be read from cache.

Why does my token count differ from the API's prompt_tokens?

The API counts the rendered prompt: your messages wrapped in the chat template, plus tool schemas in the template's own format. A tokenizer run on your raw text misses that. Use the tokenizer to compare versions, and usage.prompt_tokens for the bill.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity