BroadRouter

How to reduce LLM API costs: a 10-step checklist, in order

By the BroadNet Inference team · 8 min read · Published

To reduce LLM API costs, first log tokens per request, then work in this order: keep the prompt prefix stable so it caches, trim prompts, cap output and reasoning, use the smallest model that passes your tests, batch or shift work that can wait, cache repeated answers, bound retries, and move to a flat plan once usage passes break-even.

Most “cut your LLM bill by 70%” lists mix good advice with numbers nobody measured. This checklist is ordered by what to do first, and every figure comes from a price file or a provider’s own docs. Where a step has a deeper post on this site, it links there. BroadNet prices below come from its pricing file, synced 2026-11-14.

What drives an LLM API bill?

Four numbers decide almost every bill:

  1. New input tokens. The prompt, minus any part read from cache.
  2. Cached input tokens. The repeated prefix, billed at a lower rate.
  3. Output tokens. The answer, plus reasoning tokens on thinking models.
  4. Request count. Each call, including each retry, is billed on its own.

Output is the expensive unit. On BroadNet’s chat models, per 1M tokens:

Model Input Cached input Output
DeepSeek V4.1 Flash $0.175 $0.0055 $0.52
Qwen3.8 27B $0.175 $0.0435 $2.17
GLM 5.3 $0.78 $0.173 $3.46
Gemma 4 VL 26B $0.10 not published $0.22

So the levers are: send fewer new tokens, read more from cache, generate less, and make fewer calls. How LLM API pricing works covers each unit in detail.

The checklist at a glance

Step What you change Cuts Effort
1. Measure Logging only Nothing yet; shows where to cut Low
2. Cache the prefix Prompt order Input price Low
3. Trim the prompt Prompt content Input tokens Medium
4. Cap output and reasoning Request settings Output tokens Low
5. Right-size the model Model choice All prices Medium
6. Batch or shift work Scheduling Price per token, where offered Medium
7. Cache answers App code Whole requests Medium
8. Bound retries and loops App code Wasted requests Low
9. Set spend limits Account and keys Runaway bills Low
10. Check flat-rate break-even Billing model Total bill at high volume Low

1. Why measure before you cut anything?

Because the bill hides the cause. Two requests that look the same can differ ten times in tokens. Every chat completion returns a usage block with prompt_tokens, completion_tokens and, where the provider reports it, cached tokens. Log those per feature and per user, without the prompt text. After a week you know which feature spends the most and whether it is input, cache misses or output.

Usage logging with PII redacted shows a log schema. To size a feature before it ships, use the LLM API cost calculator or the method in estimate cost before launch.

2. How does prompt caching cut cost?

Providers cache the start of a prompt that repeats exactly. The next call reads that prefix at the cached price. OpenAI turns caching on by default for supported models, from 1,024 tokens, and says cached input is “discounted up to 95%” (checked 2026-11-15). On BroadNet, DeepSeek V4.1 Flash bills cached input at $0.0055 per 1M, against $0.175 for new input.

The rule: static content first, changing content last. System prompt, then tool schemas, then shared documents, then history, then the new question. A timestamp or a shuffled tool list at the top breaks the cache for everything after it.

3. Which prompt cuts are safe?

Cut what the model does not use. Common wins: long system prompts written as prose, more few-shot examples than the task needs, tool schemas for tools the step never calls, verbose JSON where a compact table would do, and full chat history where a summary would do. Retrieval also drifts: sending ten chunks when two answer the question is a cost and a quality problem.

Test each cut against a fixed set of inputs, so a cheaper prompt does not quietly break answers. Reduce prompt tokens has measured cuts for each of these. Language matters too: Arabic text uses more tokens per word than English on most tokenizers; see tokens per word in Arabic vs English.

4. How do you cap output and reasoning?

Ask for less, and bound it. Ask for JSON fields instead of prose, a word limit instead of “explain”, and no restating of the question. Then set max_tokens so a runaway answer stops.

Thinking models need care. BroadNet’s docs say reasoning counts toward max_tokens, so a low cap can return an empty answer. Keep 1,000 tokens or more, and set reasoning_effort to low on simple steps. On per-token keys, reasoning is billed as output. Reasoning tokens: why you pay for more than you see and the empty-answer fix cover both sides.

5. How do you pick a cheaper model without losing quality?

Compare cost per successful task, not price per token. Build 50 to 100 real inputs with expected outputs, run each candidate, and divide spend by passes. A model that is cheaper per token but fails a third of the time can cost more once you count retries and human review.

Check features first. On BroadNet, tools and structured output are documented for GLM 5.3 and Qwen3.8 27B only. Gemma 4 VL 26B is BroadNet’s smaller text-and-image model; check its price in the table above for simple tasks. Many apps use a small model for easy steps and a larger one for hard ones; you choose the model in your own code. How to evaluate an LLM before switching gives a test plan.

Do not wait for your current model to get cheaper. Providers rarely cut an existing model’s price; savings usually come with a new version. See OpenAI vs Anthropic vs Google API price history and the biggest LLM API price cuts.

6. Can you batch or shift work that can wait?

Where the provider offers it, yes, and the discounts are large. All checked 2026-11-15:

Provider Discount Catch
OpenAI Batch API 50% versus synchronous calls Completes within 24 hours; separate rate limits
Anthropic Message Batches API 50% Asynchronous; most batches finish in under an hour
DeepSeek API, off-peak Off-peak rates are half of peak Peak is 01:00 to 04:00 and 06:00 to 10:00 UTC, Monday to Friday

BroadNet Inference has no Batch API and no off-peak pricing. For overnight jobs on BroadNet, run them through a resumable runner on the normal endpoint within your key’s limits. LLM batch processing without a Batch API has the code. If the job runs for hours, a flat session may cost less than the tokens (step 10).

7. When should you cache whole answers?

When the same question arrives again with the same inputs. FAQ bots, classification of repeated items and re-runs of a pipeline after a crash all repeat work. An exact-match cache keyed on model, prompt and settings returns the stored answer and skips the call. Set an expiry so answers do not go stale. Cache LLM responses in Python shows a small implementation.

8. How do retries and loops waste money?

A retried request is a new inference and is billed again on per-token keys. Retry only on 429 and 5xx errors, with exponential backoff and jitter, and cap attempts. Never retry a 400: the same bad request fails the same way.

Agents add a second risk: a loop that keeps calling the model. Cap steps per task and stop when the same tool call repeats. Handling 429 errors and 10 ways to cut coding agent costs cover both.

9. How do you stop a bill from running away?

Alerts tell you after the fact. Limits stop it. Use a prepaid balance, one key per app so you can revoke one without stopping the rest, and an IP allow-list on server keys. In code, set a budget per job and stop when it is spent. LLM API spend control lays out the layers, from account to request.

10. When does a flat-rate plan beat per-token pricing?

When an agent runs most of the day. BroadNet sells DeepSeek V4.1 Flash per token or per session: Full Moon $60 a month, Orbit $15 a week or Meteor $1 an hour, with no token meter. One session runs one request at a time.

BroadNet’s worked scenarios on per-token billing:

Workload Per token a month Cheaper option
Side project $14.23 pay as you go
Daily assistant $42.67 pay as you go
Full-time coding agent $65.40 Full Moon
Always-on agent $196.20 Full Moon

At 80% cache hits and 20 input tokens per output token, break-even is about 917 million input and 46 million output tokens a month. Light use stays cheaper per token. The full break-even math, or try your volumes in the flat-rate vs per-token calculator.

Which steps should you skip?

Some popular advice costs more than it saves:

  • Compressing prompts until they break. Savings on input are lost if answer quality drops and users retry.
  • Summarising history every turn. Each summary is an extra call, and it changes the prefix, so the cache misses. Summarise rarely and append the rest.
  • Chasing the lowest price per token alone. Rate limits, missing features and failure rates all add cost.
  • Self-hosting for a small workload. GPUs bill by the hour whether busy or not. Self-hosting vs API cost shows where the break-even sits.

Where BroadNet Inference fits

BroadRouter is BroadNet’s own website, so weigh this section with that in mind. BroadNet Inference is an OpenAI-compatible API with four chat models priced per token, cached input on three of them, and a flat DeepSeek V4.1 Flash plan for steady agent use. It does not offer a Batch API or off-peak rates, so for large jobs that can wait, OpenAI’s or Anthropic’s batch discounts, or DeepSeek’s off-peak hours, may cost less. New accounts get $5 of starting credit, which covers Gemma 4 VL 26B, both Transcribe models and Gemma hourly keys; DeepSeek, GLM and Qwen need a top-up. Self-serve has no SLA. See pricing or compare the plans.

Questions

What is the fastest way to reduce LLM API costs?

Start with prompt caching. Put the system prompt, tool schemas and shared documents first and keep them byte-identical between calls. Cached input is billed at a fraction of new input, and you change no model or feature. Then cap output, because output tokens cost more than input tokens.

Does a Batch API really halve the price?

At OpenAI and Anthropic, yes. OpenAI's Batch API gives a 50% discount with a 24-hour completion window, and Anthropic's Message Batches API cuts cost by 50 percent (both checked 2026-11-15). BroadNet Inference has no Batch API, so run jobs through a resumable runner on normal endpoints instead.

Are reasoning tokens part of my bill?

Yes, on per-token keys. Reasoning is reported in completion tokens and billed at the output price. Set reasoning_effort to low for simple steps, but keep max_tokens at 1,000 or more so the answer is not cut off.

Is a smaller model always cheaper?

Per token, usually. Per task, not always. A smaller model that needs two retries or a longer prompt can cost more than a larger one that gets it right first time. Compare cost per successful task on your own test set.

When is a flat-rate plan cheaper than paying per token?

When usage is high and steady. On BroadNet, one Full Moon session is $60 a month for DeepSeek V4.1 Flash. At 80% cache hits, it breaks even at about 917 million input and 46 million output tokens a month. Below that, per token is cheaper.

Do retries cost money?

Yes. A retried request is a new inference and is billed again on per-token keys. Retry only on 429 and 5xx errors, with backoff, and cap the number of attempts.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity