“We would save money running it ourselves” is one of the most common claims in LLM cost planning, and one of the least often checked. The math is not hard. It has one variable that people guess wrong (utilization) and one cost they leave out (people). This post gives the formula, real rental prices, and a way to run it with your own numbers.
What does GPU rental cost right now?
Hourly prices from official pricing pages, fetched 2026-11-14 (RunPod) and 2026-11-16 (Lambda). A month here is 730 hours.
| GPU | Provider and tier (checked 2026-11-14) | Per GPU-hour | Per month, always on |
|---|---|---|---|
| H100 SXM | RunPod Secure Cloud | USD 4.39 | about USD 3,205 |
| H100 SXM | RunPod Community Cloud | USD 2.69 | about USD 1,964 |
| H100 PCIe | RunPod Secure Cloud | USD 2.89 | about USD 2,110 |
| H200 | RunPod Secure Cloud | USD 5.29 | about USD 3,862 |
| A100 80GB SXM | RunPod Secure Cloud | USD 2.19 | about USD 1,599 |
| L40S | RunPod Secure Cloud | USD 1.09 | about USD 796 |
| H100 SXM | Lambda on-demand, 1 to 8 GPUs (in effect from 2026-11-16; checked 2026-11-16) | USD 4.59 to 4.89 | about USD 3,351 to 3,570 |
Lambda’s page (checked 2026-11-16) says these prices take effect on 2026-11-16 at 00:00 UTC; the per-GPU rate falls as the node grows, to USD 4.59 on an 8-GPU node. RunPod’s Community Cloud rates are “from” prices that vary by host. Secure and Community Cloud are different hosting tiers, so read the tier terms if you have data-handling rules. Many models need more than one GPU: an 8-GPU H100 node at the Secure Cloud rate is about USD 25,638 a month.
How do you calculate self-hosted cost per million tokens?
A GPU produces tokens at some aggregate rate across all the requests it is batching. Your cost per token is the hourly price spread over the tokens it actually produces:
cost per 1M tokens = hourly price ÷ (aggregate tokens per second × 3,600 ÷ 1,000,000) ÷ utilization
Utilization is the share of hours the GPU is doing useful work at that rate. A worked example with assumed numbers (not a benchmark: measure your own model):
- GPU: one H100 SXM on Community Cloud, USD 2.69 an hour.
- Assumed aggregate output: 1,500 tokens per second with good batching.
- Tokens per hour at full load: 1,500 × 3,600 = 5.4 million.
| Utilization | Effective tokens per hour | Cost per 1M output tokens |
|---|---|---|
| 100% (always saturated) | 5.4M | USD 0.50 |
| 50% | 2.7M | USD 1.00 |
| 25% (busy business hours only) | 1.35M | USD 1.99 |
| 10% (spiky product traffic) | 0.54M | USD 4.98 |
Now compare with an API price per 1M output tokens. The LLM API pricing table shows what dozens of hosts charge for the same open weights; on BroadNet Inference, DeepSeek V4.1 Flash costs $0.52 per 1M output tokens and Gemma 4 VL 26B costs $0.22. Find the row where your self-hosted cost drops below the API price: that is your break-even utilization. The example ignores input tokens, which also take GPU time (prefill), so real self-hosted costs are higher than the table shows.
How many tokens a month do you need to break even?
Turn it around. A GPU left running costs its monthly price regardless. The API costs tokens × price. Break-even volume is:
monthly tokens to break even = monthly GPU cost ÷ API price per 1M tokens × 1,000,000
For one Community Cloud H100 (about USD 1,964 a month) against a $0.52-per-1M output price, divide 1,964 by that price to get the millions of output tokens per month you must replace before the GPU pays for itself. Do the same with your blended input and output price if your workload is input-heavy, as agents usually are; the flat-rate vs per-token post shows how to blend them.
And that GPU must actually serve the model. If the model needs four GPUs, multiply the monthly cost by four before you divide.
What costs do people leave out?
| Cost | What it covers | How to count it |
|---|---|---|
| Engineering setup | Serving engine, quantization choice, autoscaling, load tests | One-off hours × your rate |
| Ongoing ops | Upgrades, driver and engine updates, model refreshes | Hours per month × your rate |
| On-call | Someone wakes up when the node dies | Your on-call policy |
| Redundancy | A second node so one failure is not an outage | Doubles the GPU line |
| Idle headroom | Capacity for peaks you must keep warm | Lowers utilization |
| Evaluation | Checking a new model version before switching | Hours per release |
Redundancy is the one that changes the answer most. A single GPU is a single point of failure. Two GPUs halve your utilization overnight. See evaluating a model before switching for the eval work that each upgrade brings.
When is self-hosting the right call?
- Steady, high volume. Batch pipelines that keep GPUs saturated around the clock.
- Data must stay inside your network. A hard requirement, not a cost argument. Check whether an API with zero data retention meets your rules first.
- A model no API serves. Your own fine-tune or a niche open-weight model.
- Guaranteed capacity. You need throughput an API’s rate limits will not give you.
For everything else, especially spiky traffic, early products and agents that run in bursts, an API is cheaper and has no on-call rota.
Are there flat-price options between a GPU and per-token billing?
There is a middle ground between renting a GPU and paying per token. BroadNet sells an hourly key for Gemma 4 VL 26B at $0.50 an hour (one stream, 120 requests a minute, 300,000 tokens a minute), which is $360.00 if you ran it all month. It also sells unlimited plan sessions for DeepSeek V4.1 Flash at $60 a month, each running one request at a time at about 100 tokens per second. These give a predictable bill like a rented GPU, with no setup or ops, but with fixed concurrency per key or session rather than a whole GPU’s batch capacity.
Where BroadNet Inference fits
BroadNet Inference is the API side of this comparison: per-token prices, flat plans and hourly keys for open-weight models, some partner-hosted with US inference partners, behind an OpenAI-compatible API. It fits teams whose traffic would leave a rented GPU idle much of the day, or who do not want to run inference infrastructure. It does not fit if you need a custom fine-tune served, on-premises deployment, or hundreds of parallel requests on one key. Run your numbers in the LLM API cost calculator and compare with the table above. If you are still choosing between open and closed models, open-weight vs closed model API cost compares per-token prices from each provider’s own page. The self-hosting vs API calculator does the whole break-even with your measured throughput and these GPU prices.
