Self-hosting vs API cost calculator
Updated
In short: Self-hosted cost per 1M tokens = GPU price per hour × GPUs ÷ (measured tokens per second × 3,600 × utilisation ÷ 1,000,000). One H100 at RunPod's Community Cloud rate costs about USD 1,964 a month always on, so it must replace about 8,048 million tokens a month of DeepSeek V4.1 Flash to break even.
Estimates, before tax. GPU presets are on-demand list prices checked 2026-11-14; reserved, spot and committed-use prices are lower. BroadNet prices synced 2026-11-14. "Your current API" presets are each maker's standard list price on its own API (lowest prompt-size tier), read from the maker's pricing page on the date shown; batch, off-peak and volume discounts are not applied, so type your real price if it differs. The comparison assumes the self-hosted model is good enough for your task; a model you host is not the same model as the API rows, so compare quality first. Flat plans run DeepSeek V4.1 Flash only, one request at a time per session; the hourly key runs BroadNet Gemma 4 VL 26B.
Report a wrong price or number (email to info@broadnet.ai, with this page's link and your inputs filled in).
How it works
- Choose a GPU preset (official on-demand list prices from RunPod, Lambda and AWS, dated) or type your own hourly price, then the number of GPUs one model server needs.
- Enter the throughput you measured for that server, in tokens per second across all concurrent requests, and the share of the month it runs at that rate (utilisation).
- Enter your monthly volume in millions of tokens, the share that is output, the share of input served from cache, and any monthly engineering and ops cost.
- Optionally enter the API you use now: pick a maker's list price or type your own input, cached-input and output prices per 1M tokens. It gets its own row and break-even.
- Read the table: self-hosted cost per 1M tokens and per month against your API, every BroadNet per-token price and flat plan, with the break-even volume. Copy the link or download CSV.
The formula
server cost per hour = GPU price per hour × GPUs
tokens per month (cap) = tokens per second × 3,600 × 730 × utilisation
self-hosted per 1M = server cost per hour / (tokens per second × 3,600 × utilisation / 1,000,000)
servers needed = ceil(monthly tokens / tokens per month cap)
self-hosted per month = servers × server cost per hour × 730 + ops
API per month = input × (1 − cache) × input price + input × cache × cached price
+ output × output price (per 1M tokens)
break-even volume = (server cost per hour × 730 + ops) / API price per 1M
break-even utilisation = (server cost per hour × 730 + ops)
/ (tokens per second × 3,600 × 730 / 1,000,000 × API price per 1M)A month here is 730 hours (8,760 ÷ 12), and the servers are rented always-on, as most teams run a production model. Utilisation is the share of those hours the server spends at the throughput you measured; idle nights, traffic peaks you must keep headroom for and a standby node all lower it. If break-even volume is larger than one server's monthly capacity, self-hosting cannot match that API price at your throughput. See GPU-hour, tokens per second and quantization in the glossary.
GPU list prices
On-demand prices per GPU-hour from each provider's official page, checked 2026-11-14. A month is 730 hours.
| GPU | Provider and tier | Per GPU-hour | Per month, always on | Notes and source |
|---|---|---|---|---|
| H100 SXM 80GB | RunPod Secure Cloud | USD 4.39 | about USD 3,205 | Billed per second. Pricing, checked 2026-11-14. |
| H100 SXM 80GB | RunPod Community Cloud | USD 2.69 | about USD 1,964 | "From" price; varies by host. Pricing, checked 2026-11-14. |
| H100 PCIe 80GB | RunPod Secure Cloud | USD 2.89 | about USD 2,110 | Billed per second. Pricing, checked 2026-11-14. |
| H200 141GB | RunPod Secure Cloud | USD 5.29 | about USD 3,862 | Billed per second. Pricing, checked 2026-11-14. |
| B200 180GB | RunPod Secure Cloud | USD 7.99 | about USD 5,833 | Billed per second. Pricing, checked 2026-11-14. |
| A100 SXM 80GB | RunPod Secure Cloud | USD 2.19 | about USD 1,599 | Billed per second. Pricing, checked 2026-11-14. |
| L40S 48GB | RunPod Secure Cloud | USD 1.09 | about USD 796 | Billed per second. Pricing, checked 2026-11-14. |
| RTX 4090 24GB | RunPod Community Cloud | USD 0.34 | about USD 248 | "From" price; varies by host. Pricing, checked 2026-11-14. |
| H100 SXM 80GB | Lambda On-demand, 8-GPU node | USD 4.59 | about USD 3,351 | Lambda says its listed prices take effect 2026-11-16 at 00:00 UTC; plus sales tax. Pricing, checked 2026-11-14. |
| H100 SXM 80GB | Lambda On-demand, 1 GPU | USD 4.89 | about USD 3,570 | Lambda says its listed prices take effect 2026-11-16 at 00:00 UTC; plus sales tax. Pricing, checked 2026-11-14. |
| B200 180GB | Lambda On-demand, 8-GPU node | USD 7.69 | about USD 5,614 | Lambda says its listed prices take effect 2026-11-16 at 00:00 UTC; plus sales tax. Pricing, checked 2026-11-14. |
| A100 80GB | Lambda On-demand, 8-GPU node | USD 3.19 | about USD 2,329 | Lambda says its listed prices take effect 2026-11-16 at 00:00 UTC; plus sales tax. Pricing, checked 2026-11-14. |
| GH200 96GB | Lambda On-demand, 1 GPU | USD 2.59 | about USD 1,891 | Lambda says its listed prices take effect 2026-11-16 at 00:00 UTC; plus sales tax. Pricing, checked 2026-11-14. |
| H100 80GB | AWS EC2 p5.48xlarge, us-east-1 | USD 6.88 | about USD 5,022 | p5.48xlarge, 8 GPUs, USD 55.04 an hour for the instance in us-east-1 (Linux, on-demand), divided by 8. Includes the CPU, memory and storage bundled with the instance. Pricing, checked 2026-11-14. |
| H200 141GB | AWS EC2 p5en.48xlarge, us-east-1 | USD 7.91 | about USD 5,776 | p5en.48xlarge, 8 GPUs, USD 63.296 an hour for the instance in us-east-1 (Linux, on-demand), divided by 8. Includes the CPU, memory and storage bundled with the instance. Pricing, checked 2026-11-14. |
| L40S 48GB | AWS EC2 g6e.xlarge, us-east-1 | USD 1.86 | about USD 1,359 | g6e.xlarge, 1 GPU, USD 1.861 an hour for the instance in us-east-1 (Linux, on-demand), divided by 1. Includes the CPU, memory and storage bundled with the instance. Pricing, checked 2026-11-14. |
The calculator ships no throughput numbers on purpose. Tokens per second for the same GPU can differ several times over between a small quantized model with big batches and a large model serving long prompts one at a time, so a borrowed benchmark would decide the answer for you. The self-hosting vs API post walks through how to measure it.
Worked example
One H100 SXM on RunPod Community Cloud at USD 2.69 an hour costs about USD 1,964 a month. Suppose you measured 1,500 tokens per second (an assumption for this example) and the server is busy half the time: that is 1,971 million tokens a month at USD 1.00 per 1M tokens. With 80% input and 20% output, DeepSeek V4.1 Flash costs USD 0.24 per 1M tokens and BroadNet Gemma 4 VL 26B USD 0.12, so this server needs about 8,048 million and 15,836 million tokens a month to break even. Its capacity is 1,971 million, so at these numbers the API is cheaper, and a Full Moon session at $60 a month (up to about 263 million output tokens at 100 tokens per second) costs less again for a single agent that runs one request at a time.
When self-hosting is still the right call
- Data must stay inside your network, and an API with zero data retention does not meet your rules.
- You serve your own fine-tune or a model no API offers.
- You keep GPUs saturated around the clock with batch work.
- You need guaranteed capacity beyond an API's rate limits; the rate limit calculator shows what a key allows.
To price the API side in more detail, with cache hits and reasoning tokens, use the LLM API cost calculator; to compare a flat plan with per-token billing, the flat-rate vs per-token calculator. The $5 starting credit is enough to measure your own prompts on BroadNet first.
Questions
Is it cheaper to self-host an LLM than to use an API?
Only at high, steady volume. A rented GPU bills every hour whether it is busy or idle; an API bills only the tokens you use. One H100 SXM at RunPod's Community Cloud rate (USD 2.69 an hour, checked 2026-11-14) costs about USD 1,964 a month left running, so it has to replace about 8,048 million tokens a month of DeepSeek V4.1 Flash at its blended price to break even, before engineering time.
How do you calculate the cost per token of a self-hosted LLM?
Divide what the GPUs cost per hour by the tokens they produce per hour: cost per 1M tokens = GPU price per hour × number of GPUs ÷ (tokens per second × 3,600 × utilisation ÷ 1,000,000). Utilisation is the share of hours the server really runs at that speed. Add engineering and on-call time as a monthly cost on top.
How many tokens per second does an H100 generate?
There is no single number. Throughput depends on the model size, quantization, serving engine (such as vLLM or SGLang), batch size, prompt and answer lengths, and concurrency. Run your own prompts at your real concurrency and record total tokens per second across all requests. That measured figure is the one this calculator needs; vendor peak figures assume ideal batching.
How much does it cost to rent an H100 per hour?
On 2026-11-14: RunPod listed H100 SXM at USD 4.39 an hour on Secure Cloud and from USD 2.69 on Community Cloud; Lambda listed USD 4.59 to 4.89 per GPU-hour on demand, for prices taking effect 2026-11-16; AWS's p5.48xlarge (8 × H100) was USD 55.04 an hour in us-east-1, about USD 6.88 per GPU. The presets carry these prices; check the pages before you commit.
What utilisation do I need to break even with an API?
Solve the cost formula for utilisation: break-even utilisation = GPU cost per hour ÷ (tokens per second × 3,600 ÷ 1,000,000 × API price per 1M tokens). Add your monthly engineering cost divided by 730 to the GPU cost per hour if you count it. If the answer is above 100%, the server cannot beat that API price at your throughput, however busy it is. The calculator shows this for each BroadNet model and for the API you use now, if you enter its prices.
What costs do people forget when they self-host?
Engineering to set up serving and autoscaling, upgrades and model refreshes, on-call, a second node for redundancy (which halves utilisation), idle headroom for peaks, and evaluation before each model switch. Enter them as a monthly cost in the "Engineering and ops" field. Self-hosting vs API: the full cost list.
Go deeper
Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity
Sources
- BroadNet Inference machine-readable prices (pricing.json)
- RunPod GPU pricing (Secure and Community Cloud), checked 2026-11-14
- Lambda on-demand GPU pricing, checked 2026-11-14
- AWS EC2 on-demand price list, us-east-1 Linux (published 2026-11-14), checked 2026-11-14
- AWS EC2 P5 instance specs (GPUs per instance), checked 2026-11-14
- BroadNet Inference plans and typical speed
- BroadNet Inference API reference, limits