BroadRouter
Get started

BroadNet's GPU hardware and serving approach, as BroadNet describes it

By the BroadNet Inference team · 5 min read · Published · Updated

BroadNet says it runs one serving stack on three hardware tiers: data-center GPUs from A100 to B300, a workstation tier of RTX and DGX Spark cards that carries the always-on public API, and appliances on client sites. It reports about 270 models run across 13 GPU families, NVIDIA and AMD. Some API models are served by US inference partners instead.

BroadNet describes its inference hardware in unusual detail. Its model catalogue lists the GPUs it says it runs, measured throughput per card, and what it learned from running the same models across them. This page summarises what BroadNet says, in its words where it matters. We have not inspected the hardware, and the measurements are BroadNet’s own, from September 2026.

What hardware does BroadNet say it runs?

BroadNet says it runs “one fleet, three tiers, one stack”: the same engines, quantisations and gateway from data-center accelerators down to appliances on client sites.

Tier GPUs BroadNet lists What BroadNet uses it for
Data center B300, B200, H200, H100, A100 80 GB Flagship serving, quantisation and calibration, fine-tuning, dedicated private endpoints, data-residency placements
Workstation RTX PRO 6000 Blackwell, RTX 5090, RTX 3090, DGX Spark GB10, Radeon AI PRO R9700 The always-on public API, benchmarking across CUDA and ROCm, model development
Client site DGX Spark GB10, compact AMD appliances On-premise and air-gapped deployments on the identical stack

In total BroadNet reports about 270 models run, about 200 checkpoints loaded, about 40 model families, 12 or more quantisation formats and 13 GPU families. It also says “a single H100 serves the entire current public catalogue with headroom”.

Which API models run on it?

Not all of them. BroadNet’s privacy statement says its own models and the open-weight models it hosts run on BroadNet-owned or leased hardware in the United States, with partner capacity as backup, and that models served by partners run under zero-data-retention terms with US inference partners. Its Terms of Service name DeepSeek V4.1 Flash as partner-hosted.

What BroadNet’s own pages say per model:

Model on the API Hardware BroadNet names
Gemma 4 VL 26B NVIDIA Blackwell-class GPUs with a hot backup
Transcribe Arabic and Transcribe NVIDIA GPUs operated by BroadNet in the United States
DeepSeek V4.1 Flash Partner-hosted (Terms of Service §7.3)
GLM 5.3, Qwen3.8 27B Not stated for API traffic

If where your data is processed matters, ask BroadNet before you send it. Its inference page says regional processing in MENA or Europe is available on request.

How fast is each GPU?

BroadNet publishes aggregate output throughput per GPU for one model, Qwen3-VL 30B A3B Instruct at 32 concurrent requests:

GPU Tier Memory bandwidth Aggregate tokens/s
B300 Data center 8,000 GB/s 14,500
B200 Data center 8,000 GB/s 14,000
H200 Data center 4,800 GB/s 8,400
H100 Data center 3,350 GB/s 5,800
A100 80 GB Data center 2,039 GB/s 3,500
RTX PRO 6000 Blackwell Workstation 1,792 GB/s 3,100
RTX 3090 Workstation 936 GB/s 1,630
DGX Spark GB10 Workstation 273 GB/s 1,597
Radeon AI PRO R9700 Workstation 640 GB/s 1,087

BroadNet notes that the data-center figures are “planning numbers from our measured per-GPU curve”, not direct measurements on each card. Most rows rise with memory bandwidth, the usual limit for decoding, but not all: DGX Spark GB10 (273 GB/s) nearly matches the RTX 3090 (936 GB/s), and the Radeon AI PRO R9700 (640 GB/s) trails both.

What does BroadNet say its hardware taught it?

The catalogue lists eight findings, each marked “measured on our own hardware”. The ones that matter most for API buyers:

  • CUDA vs ROCm. Running the same models on NVIDIA CUDA and AMD ROCm, CUDA turned about twice as much memory bandwidth into tokens, so BroadNet places workloads accordingly.
  • Speculative decoding under load. EAGLE3 cost 69% of throughput at 32 concurrent requests, so BroadNet kept multi-token-prediction drafters. Its Uno serving study measures the same effect for another method.
  • Serving matters as much as weights. The same Gemma 4 26B weights served by a hosted provider scored 0.13 lower on average than what BroadNet calls “our own serving”. BroadNet’s rule: keep a challenger and its baseline on the same serving before comparing them.
  • A smaller active model can win. The 26B mixture of experts beat the dense Gemma 4 31B on 5 of 6 pipeline steps with about one eighth of the active parameters.
  • Frontier parity on one pipeline. In a blind evaluation of 240 traces, BroadNet’s served Gemma 4 matched Claude Sonnet on five of six measured steps, with 1.2 to 6.5 times faster responses per step.
  • Per-request speed. Qwen3-VL 30B A3B ran at 36.5 to 110 tokens per second per request on what BroadNet describes as its own serving, by BroadNet’s measurement, against 7 to 19 at four OpenRouter providers BroadNet measured.
  • Architecture beats model choice. On a hard set where two local vision models disagreed, escalating to a hosted adjudicator moved accuracy from 1 of 12 to 11 of 12.

The Sonnet comparison is the strongest claim, and it is narrow: one company’s chatbot pipeline, judged blind by three judges, on 240 production conversations from 11 tenants, per BroadNet’s Gemma 4 page. It does not mean Gemma 4 matches Sonnet in general. BroadNet has not published a full write-up of that evaluation on its research page.

How is the Gemma 4 model served?

BroadNet’s Gemma 4 VL 26B is the clearest example of its serving choices, per its model page:

  • Weights: Google’s Gemma 4 26B A4B, quantised by BroadNet to 4-bit AWQ. BroadNet tested NVFP4 and retired it after a head-to-head.
  • Decoding: multi-token-prediction speculative decoding with two drafted tokens (k=2), and an fp8 KV cache.
  • Template: a chat template BroadNet tuned for its conversational workflow.
  • Resilience: a hot backup with automatic failover.
  • Result: about 136 tokens per second on a single stream, and 90.86 on BroadNet’s internal quality battery.

See quantisation and speculative decoding for the terms.

What does this mean for you?

For a pay-as-you-go user, BroadNet’s serving choices show up as the published speed, about 100 tokens per second typical on the plan model, and in the limits: 30 requests a minute and 4 in flight per key. BroadNet’s terms say self-serve has no availability commitment; service levels come only with a written agreement. How other providers’ SLAs compare.

For larger needs, BroadNet offers private endpoints on reserved GPUs, with your model, key, limits and region, and on-premise appliances on the same stack. If you are weighing that against renting your own GPUs, our self-hosting vs API cost guide works through the arithmetic, and the models BroadNet ruled out shows how it chose what to serve.

Questions

Does BroadNet run its own GPUs?

BroadNet says it operates its own NVIDIA and AMD GPU fleet. Its privacy statement says its own models and the open-weight models it hosts run on BroadNet-owned or leased hardware in the United States, with partner capacity as backup. Partner-served models, such as DeepSeek V4.1 Flash, run with US inference partners.

Which GPU serves the public API?

BroadNet's catalogue places "the always-on public API" on its workstation tier (RTX PRO 6000 Blackwell, RTX 5090, RTX 3090, DGX Spark GB10, Radeon AI PRO R9700). Its Gemma 4 page names NVIDIA Blackwell-class GPUs with a hot backup.

Is there a published uptime SLA?

No for self-serve. BroadNet's Terms of Service say self-serve has no availability commitment, and service levels exist only in a separate written agreement, such as for a private endpoint.

Can I get a dedicated GPU?

Yes. BroadNet offers private endpoints on a reserved GPU or node, with your choice of model, your own key and limits, and the region you need. They are priced to your traffic on request.

Can BroadNet run models on my premises?

BroadNet says its client-site tier runs the same stack on DGX Spark GB10 and compact AMD appliances for on-premise and air-gapped deployments.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity