BroadRouter

Real LLM API cost by language

Updated

Sending 1M characters of Modern Standard Arabic costs $0.0077 to $6.39, depending on model and provider, against $0.0043 to $4.27 for English. We priced 618 provider rows for 108 models using 16 measured tokenizers. The cheapest Arabic input is Llama 3.1 8B Instruct on DeepInfra. Free CSV and JSON.

Download CSV Download JSON Croissant metadata CC BY 4.0 · prices checked 2026-11-14 to 2026-11-15 · BroadNet prices 2026-11-14 · tokens measured 2026-11-15

BroadRouter is BroadNet's own site. BroadNet rows come from its pricing.json and are labelled BroadNet; every other price is read from that provider's own page. Where another provider is cheaper, the tables say so.

Key findings

  1. We priced 614 standard-tier prices from 35 providers, plus 4 BroadNet rows, for 108 models whose tokenizer we measured (16 tokenizers). Each model is mapped to its tokenizer with recorded evidence (the official repo, a byte-identical copy or OpenAI's tiktoken table); BroadNet's Gemma build is assumed to keep Google's tokenizer.
  2. Sending 1M characters of Modern Standard Arabic costs $0.0077 to $6.39 depending on model and provider; English costs $0.0043 to $4.27. The cheapest Arabic input is Llama 3.1 8B Instruct on DeepInfra, and the cheapest English input is Llama 3.1 8B Instruct on DeepInfra.
  3. The cheapest place to send English text is also the cheapest for 12 of the other 14 languages and content kinds. For Hindi and Urdu, the language changes the winner: OpenAI GPT OSS 120B on CoreWeave Forge.
  4. Language tax, in money: on the same 2,009 FLORES-200 sentences, Modern Standard Arabic needs 25% (Ministral 3 tokenizer, 95% interval 24% to 25%) to 203% (Phi-4, 201% to 205%) more tokens than English, so it costs that much more on the same model. For content that is 1M words in English, the median row charges $0.192 more to send it in Arabic and $0.809 more to generate it.
  5. The largest tax in this data is Hindi on the Phi-4 tokenizer: 378% more tokens than the same English sentences (95% interval 374% to 382%). Hindi needs 31% (Gemma 4) to 378% (Phi-4) more; the median row charges $0.644 more to send Hindi content that is 1M words in English.
  6. For Arabic MSA, the lowest price per token (Llama 3.1 8B Instruct on DeepInfra) is also the cheapest per character, at $0.0077 per 1M characters. The tokenizer with the fewest Arabic MSA tokens is Ministral 3, yet its cheapest row (Ministral 3 3B on Amazon Bedrock) costs $0.0298: a low enough per-token price beats a lean tokenizer. Priced per character, 150 pairs of models swap order. Example among the 20 cheapest models per token: Llama 3.2 1B Instruct on Cloudflare Workers AI costs less per token than OpenAI GPT OSS 120B on CoreWeave Forge ($0.0270 against $0.0300), but $0.0104 against $0.0096 per 1M characters of Arabic MSA.
  7. The lowest price per token is Llama 3.1 8B Instruct on DeepInfra ($0.0200 per 1M input tokens), but for Hindi it is not the cheapest: OpenAI GPT OSS 120B on CoreWeave Forge sends 1M characters for $0.0097 against $0.0104, because the gpt-oss o200k_harmony tokenizer needs 38% fewer tokens for the same Hindi text. The tokenizer with the fewest Hindi tokens is Gemma 4, yet its cheapest row (Gemma 4 26B A4B on DeepInfra) costs $0.0190: a low enough per-token price beats a lean tokenizer. Priced per character, 463 pairs of models swap order. Example among the 20 cheapest models per token: Qwen3-8B on SiliconFlow costs less per token than Gemma 4 26B A4B on DeepInfra ($0.0600 against $0.0700), but $0.0558 against $0.0190 per 1M characters of Hindi.
  8. BroadNet, which runs this site, is not the cheapest input row for any language here. Its cheapest Arabic MSA row is BroadNet Gemma 4 VL 26B at $0.0343 per 1M characters.

What is the cheapest way to send or generate each language?

Cheapest standard-tier row for 1M characters of each language. Input is what you send (prompts, documents); output is what the model writes. Promotional prices are not ranked.

LanguageCheapest inputInput per 1M charsPer 1,000 wordsCheapest outputOutput per 1M charsDearest input
EnglishLlama 3.1 8B Instruct on DeepInfra$0.0043$0.000026Llama 3.1 8B Instruct on DeepInfra$0.0087$4.27
Arabic MSALlama 3.1 8B Instruct on DeepInfra$0.0077$0.000046Llama 3.1 8B Instruct on DeepInfra$0.0155$6.39
Gulf ArabicLlama 3.1 8B Instruct on DeepInfra$0.0086$0.000048Llama 3.1 8B Instruct on DeepInfra$0.0171$7.34
Egyptian ArabicLlama 3.1 8B Instruct on DeepInfra$0.0079$0.000046Llama 3.1 8B Instruct on DeepInfra$0.0157$6.66
Levantine ArabicLlama 3.1 8B Instruct on DeepInfra$0.0080$0.000048Llama 3.1 8B Instruct on DeepInfra$0.0160$6.71
Najdi ArabicLlama 3.1 8B Instruct on DeepInfra$0.0078$0.000046Llama 3.1 8B Instruct on DeepInfra$0.0155$6.40
Mesopotamian ArabicLlama 3.1 8B Instruct on DeepInfra$0.0079$0.000048Llama 3.1 8B Instruct on DeepInfra$0.0157$6.54
ArabiziLlama 3.1 8B Instruct on DeepInfra$0.0083$0.000045Llama 3.1 8B Instruct on DeepInfra$0.0166$7.75
HindiOpenAI GPT OSS 120B on CoreWeave Forge$0.0097$0.000049Llama 3.1 8B Instruct on DeepInfra$0.0209$6.44
UrduOpenAI GPT OSS 120B on CoreWeave Forge$0.0102$0.000048Gemma 3 4B on Amazon Bedrock$0.0254$6.83
ChineseLlama 3.1 8B Instruct on DeepInfra$0.0160$0.000031Llama 3.1 8B Instruct on DeepInfra$0.0320$15.43
FrenchLlama 3.1 8B Instruct on DeepInfra$0.0055$0.000034Llama 3.1 8B Instruct on DeepInfra$0.0110$4.65
SpanishLlama 3.1 8B Instruct on DeepInfra$0.0053$0.000032Llama 3.1 8B Instruct on DeepInfra$0.0107$4.51
CodeLlama 3.1 8B Instruct on DeepInfra$0.0051$0.000039Llama 3.1 8B Instruct on DeepInfra$0.0102$5.08
JSONLlama 3.1 8B Instruct on DeepInfra$0.0063$0.000078Llama 3.1 8B Instruct on DeepInfra$0.0126$6.29

How much is the language tax in money?

Same meaning, different language: the same 2,009 FLORES-200 sentences (dev and devtest) in English and each language, or Tatoeba sentence pairs where FLORES-200 has no set (Gulf Arabic, Arabizi). Extra tokens = tokens in the language ÷ tokens of the English translation, by tokenizer: the lowest and highest tokenizer, each with its 95% bootstrap interval underneath. Extra cost is for content that is 1M words in English, on the same model and provider: the median over 592 rows, then the lowest and highest.

LanguageSentencesExtra tokensMedian extra, inputMedian extra, outputLowest and highest extra (input)
Arabic MSA2,009 FLORES-20025% to 203% (95% interval 24% to 25% · 201% to 205%)$0.192$0.809$0.0139 (GPT OSS 20B on CoreWeave Forge) to $9.27 (o3 Pro on OpenAI)
Gulf Arabic49 Tatoeba15% to 154% (95% interval 8% to 23% · 137% to 171%)$0.137$0.615$0.0098 (NVIDIA Nemotron 3 Nano 30B A3B on Novita AI) to $7.69 (o3 Pro on OpenAI)
Egyptian Arabic2,009 FLORES-20029% to 195% (95% interval 28% to 30% · 193% to 198%)$0.199$0.872$0.0148 (GPT OSS 20B on CoreWeave Forge) to $9.90 (o3 Pro on OpenAI)
Levantine Arabic2,009 FLORES-20024% to 182% (95% interval 23% to 24% · 180% to 184%)$0.179$0.775$0.0128 (GPT OSS 20B on CoreWeave Forge) to $8.53 (o3 Pro on OpenAI)
Najdi Arabic2,009 FLORES-20025% to 203% (95% interval 24% to 26% · 201% to 206%)$0.195$0.825$0.0141 (GPT OSS 20B on CoreWeave Forge) to $9.38 (o3 Pro on OpenAI)
Mesopotamian Arabic2,009 FLORES-20026% to 198% (95% interval 25% to 27% · 196% to 200%)$0.195$0.841$0.0140 (GPT OSS 20B on CoreWeave Forge) to $9.37 (o3 Pro on OpenAI)
Arabizi39 Tatoeba30% to 55% (95% interval 21% to 39% · 44% to 66%)$0.183$0.752$0.0122 (Llama 3.1 8B Instruct on DeepInfra) to $10.69 (o3 Pro on OpenAI)
Hindi2,009 FLORES-20031% to 378% (95% interval 31% to 32% · 374% to 382%)$0.644$2.65$0.0154 (Gemma 3 4B on Amazon Bedrock) to $13.96 (o3 Pro on OpenAI)
Urdu2,009 FLORES-20053% to 336% (95% interval 52% to 54% · 333% to 340%)$0.503$2.29$0.0240 (GPT OSS 20B on CoreWeave Forge) to $15.99 (o3 Pro on OpenAI)
Chinese2,009 FLORES-200−12% to 88% (95% interval −13% to −11% · 86% to 90%)−$0.0149−$0.0575−$0.549 (Kimi K3 on Venice) to $6.24 (o3 Pro on OpenAI)
French2,009 FLORES-20029% to 72% (95% interval 28% to 30% · 71% to 74%)$0.182$0.787$0.0130 (GPT OSS 20B on CoreWeave Forge) to $8.69 (o3 Pro on OpenAI)
Spanish2,009 FLORES-20025% to 69% (95% interval 24% to 26% · 68% to 71%)$0.163$0.726$0.0115 (GPT OSS 20B on CoreWeave Forge) to $7.68 (o3 Pro on OpenAI)

A negative figure means the language is cheaper than English on that tokenizer (Chinese packs more meaning per token on some). Per 1M English words of content. The Gulf Arabic and Arabizi sets are small, hence their wide intervals. Najdi Arabic in FLORES-200 is mostly identical to the MSA sentences, so it lands near MSA. Every tokenizer's ratio and interval is in the tokenizer efficiency dataset.

When is the token-cheapest model not the money-cheapest?

For Arabic MSA, the lowest price per token (Llama 3.1 8B Instruct on DeepInfra) is also the cheapest per character, at $0.0077 per 1M characters. The tokenizer with the fewest Arabic MSA tokens is Ministral 3, yet its cheapest row (Ministral 3 3B on Amazon Bedrock) costs $0.0298: a low enough per-token price beats a lean tokenizer. Priced per character, 150 pairs of models swap order. Example among the 20 cheapest models per token: Llama 3.2 1B Instruct on Cloudflare Workers AI costs less per token than OpenAI GPT OSS 120B on CoreWeave Forge ($0.0270 against $0.0300), but $0.0104 against $0.0096 per 1M characters of Arabic MSA.

The lowest price per token is Llama 3.1 8B Instruct on DeepInfra ($0.0200 per 1M input tokens), but for Hindi it is not the cheapest: OpenAI GPT OSS 120B on CoreWeave Forge sends 1M characters for $0.0097 against $0.0104, because the gpt-oss o200k_harmony tokenizer needs 38% fewer tokens for the same Hindi text. The tokenizer with the fewest Hindi tokens is Gemma 4, yet its cheapest row (Gemma 4 26B A4B on DeepInfra) costs $0.0190: a low enough per-token price beats a lean tokenizer. Priced per character, 463 pairs of models swap order. Example among the 20 cheapest models per token: Qwen3-8B on SiliconFlow costs less per token than Gemma 4 26B A4B on DeepInfra ($0.0600 against $0.0700), but $0.0558 against $0.0190 per 1M characters of Hindi.

"Swap order" counts pairs of models, each at its cheapest standard input price, where one is cheaper per token but dearer per character of that language (108 models compared).

Which model families are covered?

One row per measured tokenizer. Cheapest input per 1M characters, any provider.

TokenizerModelsPrice rowsEnglishArabic MSAHindi
DeepSeek V3/V4 · DeepSeek12113$0.0129 DeepSeek V4 Flash 0731 on DeepInfra$0.0226 DeepSeek V4 Flash 0731 on DeepInfra$0.0347 DeepSeek V4 Flash 0731 on DeepInfra
GLM 5 · Z.ai586$0.0260 GLM 5.3 Flash on CoreWeave Forge$0.0496 GLM 5.3 Flash on CoreWeave Forge$0.107 GLM 5.3 Flash on CoreWeave Forge
Qwen3.5 to 3.8 · Qwen968$0.0087 Qwen3.8 27B on Chutes$0.0121 Qwen3.8 27B on Chutes$0.0165 Qwen3.8 27B on Chutes
Gemma 4 · Google324$0.0153 Gemma 4 26B A4B on DeepInfra$0.0240 Gemma 4 26B A4B on DeepInfra$0.0190 Gemma 4 26B A4B on DeepInfra
OpenAI o200k_base · OpenAI2237$0.0107 GPT-5 Nano on Microsoft Azure$0.0160 GPT-5 Nano on Microsoft Azure$0.0161 GPT-5 Nano on Microsoft Azure
gpt-oss o200k_harmony · OpenAI435$0.0064 OpenAI GPT OSS 120B on CoreWeave Forge$0.0096 OpenAI GPT OSS 120B on CoreWeave Forge$0.0097 OpenAI GPT OSS 120B on CoreWeave Forge
Llama 3 · Meta735$0.0043 Llama 3.1 8B Instruct on DeepInfra$0.0077 Llama 3.1 8B Instruct on DeepInfra$0.0104 Llama 3.1 8B Instruct on DeepInfra
Llama 4 · Meta213$0.0170 Llama 4 Scout 17B 16E Instruct on GMI Cloud$0.0296 Llama 4 Scout 17B 16E Instruct on GMI Cloud$0.0272 Llama 4 Scout 17B 16E Instruct on GMI Cloud
Kimi · Moonshot AI874$0.0848 Kimi K2.6 on Chutes$0.187 Kimi K2.6 on Chutes$0.221 Kimi K2.6 on Chutes
MiniMax M2/M3 · MiniMax532$0.0314 MiniMax M3.1 Flash on DigitalOcean$0.0464 MiniMax M3.1 Flash on DigitalOcean$0.0890 MiniMax M3.1 Flash on DigitalOcean
Gemma 3 · Google313$0.0087 Gemma 3 4B on Amazon Bedrock$0.0137 Gemma 3 4B on Amazon Bedrock$0.0109 Gemma 3 4B on Amazon Bedrock
Phi-4 · Microsoft13$0.0152 Phi-4 on DeepInfra$0.0495 Phi-4 on DeepInfra$0.0691 Phi-4 on DeepInfra
Mistral Small 3.1 · Mistral AI26$0.0109 Mistral-Small-24B-Instruct-2501 on DeepInfra$0.0152 Mistral-Small-24B-Instruct-2501 on DeepInfra$0.0293 Mistral-Small-24B-Instruct-2501 on DeepInfra
Ministral 3 · Mistral AI48$0.0219 Ministral 3 3B on Amazon Bedrock$0.0298 Ministral 3 3B on Amazon Bedrock$0.0381 Ministral 3 3B on Amazon Bedrock
Nemotron 3 · NVIDIA420$0.0109 NVIDIA Nemotron 3.5 Lightning 30B A3B on Crusoe Cloud$0.0149 NVIDIA Nemotron 3.5 Lightning 30B A3B on Crusoe Cloud$0.0190 NVIDIA Nemotron 3.5 Lightning 30B A3B on Crusoe Cloud
Qwen3 · Qwen1751$0.0112 Qwen3 30B A3B (fp8) on Cloudflare Workers AI$0.0197 Qwen3 30B A3B (fp8) on Cloudflare Workers AI$0.0475 Qwen3 30B A3B (fp8) on Cloudflare Workers AI

What does each model cost per language?

One row per model at its cheapest standard input price, plus every BroadNet row (labelled). Input cost per 1M characters for English, Arabic MSA, Hindi, Chinese and code; "Per 1M tokens" is the input price. Every provider's row, all 13 languages and content kinds, output prices and cost per 1,000 words are in the CSV and JSON.

DeepSeek V3/V4 tokenizer · 12 models, 113 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
DeepSeek V4 Flash 0731 · DeepInfra$0.0600$0.0129$0.0226$0.0347$0.0352$0.0158
DeepSeek V4 Flash · DeepInfra$0.0900$0.0193$0.0339$0.0520$0.0528$0.0238
DeepSeek V4.1 Flash · CoreWeave Forge$0.130$0.0279$0.0489$0.0751$0.0762$0.0343
DeepSeek V4.1 Flash · BroadNet BroadNet$0.175$0.0375$0.0658$0.101$0.103$0.0462
DeepSeek V4.1 Flash 1017 · Featherless$0.225$0.0482$0.0846$0.130$0.132$0.0594
DeepSeek V3.1 · DeepInfra$0.250$0.0536$0.0940$0.144$0.147$0.0660
DeepSeek V3.2 · DeepInfra$0.260$0.0557$0.0978$0.150$0.152$0.0686
DeepSeek-V3.1-Terminus · GMI Cloud$0.270$0.0578$0.102$0.156$0.158$0.0713
DeepSeek-V3.2-Exp · GMI Cloud$0.270$0.0578$0.102$0.156$0.158$0.0713
DeepSeek R1 0528 · Featherless$0.385$0.0825$0.145$0.222$0.226$0.102
DeepSeek-V4-Flash-Vision-Exp · GMI Cloud$0.440$0.0943$0.165$0.254$0.258$0.116
DeepSeek V4 Pro 0813 · Chutes$0.850$0.182$0.320$0.491$0.498$0.224
DeepSeek V4 Pro · DeepInfra$1.30$0.279$0.489$0.751$0.762$0.343
GLM 5 tokenizer · 5 models, 86 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
GLM 5.3 Flash · CoreWeave Forge$0.120$0.0260$0.0496$0.107$0.0712$0.0305
GLM 5.2 · CoreWeave Forge$0.620$0.135$0.256$0.555$0.368$0.158
GLM 5.3 · BroadNet BroadNet$0.780$0.169$0.322$0.698$0.463$0.198
GLM 5.1 · Chutes$0.900$0.195$0.372$0.805$0.534$0.229
GLM 5 · Featherless$0.950$0.206$0.393$0.850$0.564$0.242
GLM 5.3 · Chutes$1.23$0.266$0.506$1.10$0.727$0.312
Qwen3.5 to 3.8 tokenizer · 9 models, 68 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Qwen3.8 27B · Chutes$0.0390$0.0087$0.0121$0.0165$0.0226$0.0109
Qwen3.6 35B A3B · CoreWeave Forge$0.100$0.0222$0.0310$0.0423$0.0579$0.0278
Qwen3.5 9B · DeepInfra$0.100$0.0222$0.0310$0.0423$0.0579$0.0278
Qwen3.5 35B A3B · DeepInfra$0.140$0.0311$0.0434$0.0592$0.0810$0.0390
Qwen3.8 27B · BroadNet BroadNet$0.175$0.0388$0.0542$0.0740$0.101$0.0487
Qwen3.5 27B · Chutes$0.200$0.0444$0.0620$0.0846$0.116$0.0556
Qwen3.6 27B · Chutes$0.250$0.0555$0.0775$0.106$0.145$0.0696
Qwen3.5 122B A10B · SiliconFlow$0.260$0.0577$0.0806$0.110$0.150$0.0723
Qwen3.5 397B A17B · Chutes$0.450$0.0999$0.139$0.190$0.260$0.125
Qwen3.8 2.4T A95B · Featherless$1.00$0.222$0.310$0.423$0.579$0.278
Gemma 4 tokenizer · 3 models, 24 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Gemma 4 26B A4B · DeepInfra$0.0700$0.0153$0.0240$0.0190$0.0472$0.0204
BroadNet Gemma 4 VL 26B · BroadNet BroadNet (tokenizer assumed: Google's base model)$0.100$0.0219$0.0343$0.0272$0.0674$0.0291
Gemma 4 31B · CoreWeave Forge$0.100$0.0219$0.0343$0.0272$0.0674$0.0291
OpenAI o200k_base tokenizer · 22 models, 37 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
GPT-5 Nano · Microsoft Azure$0.0500$0.0107$0.0160$0.0161$0.0386$0.0127
GPT-4.1 nano (2025-04-14) · Microsoft Azure$0.100$0.0213$0.0319$0.0322$0.0771$0.0254
GPT-4o mini · Microsoft Azure$0.150$0.0320$0.0479$0.0483$0.116$0.0381
GPT-5 Mini · Microsoft Azure$0.250$0.0533$0.0798$0.0805$0.193$0.0635
GPT-4.1 Mini · Microsoft Azure$0.400$0.0854$0.128$0.129$0.309$0.102
o1-mini (2024-09-12) · Microsoft Azure$1.10$0.235$0.351$0.354$0.848$0.279
o3-mini (2025-01-31) · Microsoft Azure$1.10$0.235$0.351$0.354$0.848$0.279
o4-mini (2025-04-16) · Microsoft Azure$1.10$0.235$0.351$0.354$0.848$0.279
GPT-5 (2025-08-07) · Microsoft Azure$1.25$0.267$0.399$0.403$0.964$0.318
GPT-5 Chat · Microsoft Azure$1.25$0.267$0.399$0.403$0.964$0.318
GPT-5 Codex · Microsoft Azure$1.25$0.267$0.399$0.403$0.964$0.318
GPT-5 Search API · OpenAI$1.25$0.267$0.399$0.403$0.964$0.318
GPT-4.1 (2025-04-14) · Microsoft Azure$2.00$0.427$0.639$0.644$1.54$0.508
o3 (2025-04-16) · Microsoft Azure$2.00$0.427$0.639$0.644$1.54$0.508
GPT-4o (2024-08-06) · Microsoft Azure$2.50$0.533$0.798$0.805$1.93$0.635
GPT-4o (2024-11-20) · Microsoft Azure$2.50$0.533$0.798$0.805$1.93$0.635
GPT-4o · DigitalOcean$2.50$0.533$0.798$0.805$1.93$0.635
GPT-4o (2024-05-13) · Microsoft Azure$5.00$1.07$1.60$1.61$3.86$1.27
o3 Deep Research · Microsoft Azure$10.00$2.13$3.19$3.22$7.71$2.54
GPT-5 Pro · Microsoft Azure$15.00$3.20$4.79$4.83$11.57$3.81
o1 (2024-12-17) · Microsoft Azure$15.00$3.20$4.79$4.83$11.57$3.81
o3 Pro · OpenAI$20.00$4.27$6.39$6.44$15.43$5.08
gpt-oss o200k_harmony tokenizer · 4 models, 35 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
OpenAI GPT OSS 120B · CoreWeave Forge$0.0300$0.0064$0.0096$0.0097$0.0231$0.0076
GPT OSS 20B · CoreWeave Forge$0.0300$0.0064$0.0096$0.0097$0.0231$0.0076
GPT OSS Safeguard 20B · Amazon Bedrock$0.0700$0.0149$0.0224$0.0225$0.0540$0.0178
GPT OSS Safeguard 120B · Amazon Bedrock$0.150$0.0320$0.0479$0.0483$0.116$0.0381
Llama 3 tokenizer · 7 models, 35 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Llama 3.1 8B Instruct · DeepInfra$0.0200$0.0043$0.0077$0.0104$0.0160$0.0051
Llama 3.2 1B Instruct · Cloudflare Workers AI$0.0270$0.0059$0.0104$0.0141$0.0216$0.0069
Llama 3.2 3B Instruct · Featherless$0.0509$0.0110$0.0197$0.0266$0.0407$0.0130
Llama 3 8B Instruct · Featherless$0.0925$0.0201$0.0358$0.0483$0.0740$0.0235
Llama 3.3 70B · DeepInfra$0.100$0.0217$0.0387$0.0522$0.0800$0.0254
Llama 3.1 Instruct (70B) · Cloudflare Workers AI$0.293$0.0635$0.113$0.153$0.234$0.0746
Llama 3 Instruct (70B) · Amazon Bedrock$2.65$0.575$1.02$1.38$2.12$0.674
Llama 4 tokenizer · 2 models, 13 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Llama 4 Scout 17B 16E Instruct · GMI Cloud$0.0800$0.0170$0.0296$0.0272$0.0530$0.0204
Llama 4 Maverick 17B 128E Instruct FP8 · Amazon Bedrock$0.240$0.0510$0.0889$0.0815$0.159$0.0613
Kimi tokenizer · 8 models, 74 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Kimi K2.6 · Chutes$0.390$0.0848$0.187$0.221$0.213$0.101
Kimi K2.5 · SiliconFlow$0.450$0.0978$0.216$0.255$0.245$0.117
Kimi K2 Instruct · Novita AI$0.570$0.124$0.273$0.324$0.311$0.148
Kimi K2 Thinking · Amazon Bedrock$0.600$0.130$0.288$0.341$0.327$0.156
Kimi K2.7 Code · CoreWeave Forge$0.600$0.130$0.288$0.341$0.327$0.156
Kimi K2 Instruct 0905 · Featherless$0.600$0.130$0.288$0.341$0.327$0.156
Kimi K3 · Chutes$1.85$0.402$0.887$1.05$1.01$0.480
Kimi K3.1 Pro · Alibaba Cloud$2.00$0.435$0.959$1.14$1.09$0.519
MiniMax M2/M3 tokenizer · 5 models, 32 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
MiniMax M3.1 Flash · DigitalOcean$0.150$0.0314$0.0464$0.0890$0.0837$0.0387
MiniMax M3 · CoreWeave Forge$0.230$0.0482$0.0712$0.137$0.128$0.0593
MiniMax M2.5 · Venice$0.270$0.0565$0.0836$0.160$0.151$0.0696
MiniMax M2 · Featherless$0.278$0.0581$0.0859$0.165$0.155$0.0716
Minimax M2.1 · Amazon Bedrock$0.300$0.0628$0.0929$0.178$0.167$0.0774
Gemma 3 tokenizer · 3 models, 13 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Gemma 3 4B · Amazon Bedrock$0.0400$0.0087$0.0137$0.0109$0.0270$0.0117
Gemma 3 12B · DeepInfra$0.0500$0.0109$0.0172$0.0136$0.0337$0.0146
Gemma 3 27B · DeepInfra$0.0800$0.0175$0.0275$0.0217$0.0539$0.0233
Phi-4 tokenizer · 1 models, 3 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Phi-4 · DeepInfra$0.0700$0.0152$0.0495$0.0691$0.0815$0.0178
Mistral Small 3.1 tokenizer · 2 models, 6 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Mistral-Small-24B-Instruct-2501 · DeepInfra$0.0500$0.0109$0.0152$0.0293$0.0451$0.0136
Mistral Small 3.1 24B Instruct · Google Vertex AI$0.100$0.0219$0.0303$0.0585$0.0901$0.0271
Ministral 3 tokenizer · 4 models, 8 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Ministral 3 3B · Amazon Bedrock$0.100$0.0219$0.0298$0.0381$0.0901$0.0269
Ministral 3 8B · Amazon Bedrock$0.150$0.0328$0.0447$0.0571$0.135$0.0404
Ministral 3 14B · Amazon Bedrock$0.200$0.0438$0.0596$0.0761$0.180$0.0539
Devstral 2 123B · Amazon Bedrock$0.400$0.0876$0.119$0.152$0.360$0.108
Nemotron 3 tokenizer · 4 models, 20 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
NVIDIA Nemotron 3.5 Lightning 30B A3B · Crusoe Cloud$0.0500$0.0109$0.0149$0.0190$0.0451$0.0135
NVIDIA Nemotron 3 Nano 30B A3B · Novita AI$0.0500$0.0109$0.0149$0.0190$0.0451$0.0135
NVIDIA Nemotron 3 Super 120B A12B · DeepInfra$0.0750$0.0164$0.0224$0.0286$0.0676$0.0202
NVIDIA Nemotron 3 Ultra · CoreWeave Forge$0.500$0.109$0.149$0.190$0.451$0.135
Qwen3 tokenizer · 17 models, 51 provider rows
Model · cheapest providerInput per 1M tokensEnglishArabic MSAHindiChineseCode
Qwen3 30B A3B (fp8) · Cloudflare Workers AI$0.0510$0.0112$0.0197$0.0475$0.0326$0.0133
Qwen3-8B · SiliconFlow$0.0600$0.0132$0.0232$0.0558$0.0384$0.0156
Qwen3-14B · SiliconFlow$0.0700$0.0154$0.0271$0.0651$0.0448$0.0182
Qwen3 Coder 30B A3B · SiliconFlow$0.0700$0.0154$0.0271$0.0651$0.0448$0.0182
Qwen3 32B FP8 · DeepInfra$0.0800$0.0176$0.0310$0.0745$0.0512$0.0208
Qwen3 235B A22B Instruct 2507 FP8 · DeepInfra$0.0900$0.0198$0.0348$0.0838$0.0576$0.0234
Qwen3 Next 80B A3B Instruct · DeepInfra$0.0900$0.0198$0.0348$0.0838$0.0576$0.0234
Qwen3-30B-A3B-Instruct-2507 · SiliconFlow$0.0900$0.0198$0.0348$0.0838$0.0576$0.0234
Qwen3 Coder Next · Parasail$0.120$0.0264$0.0464$0.112$0.0768$0.0312
Qwen3 VL 30B A3B Instruct · DeepInfra$0.150$0.0329$0.0581$0.140$0.0960$0.0390
Qwen3 Next 80B A3B Thinking · GMI Cloud$0.150$0.0329$0.0581$0.140$0.0960$0.0390
Qwen3-VL 8B · SiliconFlow$0.180$0.0395$0.0697$0.168$0.115$0.0468
Qwen3 VL 235B A22B · DeepInfra$0.200$0.0439$0.0774$0.186$0.128$0.0520
Qwen3-VL-32B-Instruct · SiliconFlow$0.200$0.0439$0.0774$0.186$0.128$0.0520
Qwen3-VL-32B-Thinking · SiliconFlow$0.200$0.0439$0.0774$0.186$0.128$0.0520
Qwen3 Coder 480B A35B · Google Vertex AI$0.220$0.0483$0.0851$0.205$0.141$0.0572
Qwen3-VL-30B-A3B-Thinking · SiliconFlow$0.290$0.0637$0.112$0.270$0.186$0.0754

How are models matched to tokenizers?

A script (scripts/dataset-tokenizer-models.mjs) opens each model's own Hugging Face repo at its current commit and accepts the match only if the tokenizer file is byte-identical to the measured one, or has the same vocabulary, merges, normalizer and pre-tokenizer and gives identical counts on all 52 texts (including every FLORES-200 sentence). OpenAI's hosted models use the encoding OpenAI's tiktoken library names for them. Checked 2026-11-15: 107 models matched, 14 candidates rejected.

TokenizerMeasured fileModels matched, and how
DeepSeek V3/V4deepseek-ai/DeepSeek-V4.1-Flash @ 314113fthe measured tokenizer itself: deepseek-v4.1-flash. byte-identical tokenizer file: deepseek-v4.1-flash-1017, deepseek-v4-flash-vision-exp. same vocabulary, merges and pre-tokenizer; identical counts on every text: deepseek-v4-flash, deepseek-v4-flash-0731, deepseek-v4-pro, deepseek-v4-pro-0813, deepseek-v3.2, deepseek-v3.2-exp, deepseek-v3.1-terminus, deepseek-chat-v3.1, deepseek-r1-0528.
GLM 5zai-org/GLM-5.3 @ ba32f8cthe measured tokenizer itself: glm-5.3. byte-identical tokenizer file: glm-5.3-flash, glm-5.2, glm-5.1, glm-5.
Qwen3.5 to 3.8Qwen/Qwen3.8-27B @ 3756181the measured tokenizer itself: qwen3.8-27b. byte-identical tokenizer file: qwen3.8-2.4t-a95b. same vocabulary, merges and pre-tokenizer; identical counts on every text: qwen3.6-35b-a3b, qwen3.6-27b, qwen3.5-397b-a17b, qwen3.5-122b-a10b, qwen3.5-35b-a3b, qwen3.5-27b, qwen3.5-9b.
Gemma 4google/gemma-4-26B-A4B-it @ 4d7ae49the measured tokenizer itself: gemma-4-26b-a4b-it. byte-identical tokenizer file: gemma-4-31b-it.
OpenAI o200k_baseo200k_baseOpenAI's tiktoken model table: gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-4o, gpt-4o-2024-05-13, gpt-4o-2024-08-06, gpt-4o-2024-11-20, gpt-4o-mini, gpt-5, gpt-5-chat, gpt-5-codex, gpt-5-mini, gpt-5-nano, gpt-5-pro, gpt-5-search-api, o1, o1-mini, o3, o3-mini, o3-pro, o3-deep-research, o4-mini.
gpt-oss o200k_harmonyopenai/gpt-oss-120b @ b5c939dthe measured tokenizer itself: gpt-oss-120b. byte-identical tokenizer file: gpt-oss-20b, gpt-oss-safeguard-120b, gpt-oss-safeguard-20b.
Llama 3meta-llama/Llama-3.3-70B-Instruct @ 6f6073bthe measured tokenizer itself: llama-3.3-70b-instruct. same vocabulary, merges and pre-tokenizer; identical counts on every text: llama-3.1-8b-instruct, llama-3.1-70b-instruct, llama-3.2-1b-instruct, llama-3.2-3b-instruct, llama-3-8b-instruct, llama-3-70b-instruct.
Llama 4meta-llama/Llama-4-Maverick-17B-128E-Instruct @ 73d1471the measured tokenizer itself: llama-4-maverick. byte-identical tokenizer file: llama-4-scout.
Kimimoonshotai/Kimi-K3 @ f831ab6the measured tokenizer itself: kimi-k3. byte-identical tokenizer file: kimi-k3.1-pro, kimi-k2.7-code, kimi-k2.6, kimi-k2.5, kimi-k2-thinking, kimi-k2-0905, kimi-k2.
MiniMax M2/M3MiniMaxAI/MiniMax-M3 @ d95e30cthe measured tokenizer itself: minimax-m3. byte-identical tokenizer file: minimax-m3.1-flash. same vocabulary, merges and pre-tokenizer; identical counts on every text: minimax-m2.5, minimax-m2.1, minimax-m2.
Gemma 3google/gemma-3-27b-it @ 005ad34the measured tokenizer itself: gemma-3-27b-it. byte-identical tokenizer file: gemma-3-12b-it, gemma-3-4b-it.
Phi-4microsoft/phi-4 @ 2db69c1the measured tokenizer itself: phi-4.
Mistral Small 3.1mistralai/Mistral-Small-3.1-24B-Instruct-2503 @ 68faf51the measured tokenizer itself: mistral-small-3.1-24b-instruct. byte-identical tokenizer file: mistral-small-24b-instruct-2501.
Ministral 3mistralai/Ministral-3-14B-Instruct-2512 @ 29439f8the measured tokenizer itself: ministral-14b-2512. byte-identical tokenizer file: ministral-8b-2512, ministral-3b-2512, devstral-2512.
Nemotron 3nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 @ 77df655the measured tokenizer itself: nemotron-3-ultra-550b-a55b. byte-identical tokenizer file: nemotron-3-super-120b-a12b, nemotron-3.5-lightning. same vocabulary, merges and pre-tokenizer; identical counts on every text: nemotron-3-nano-30b-a3b.
Qwen3Qwen/Qwen3-32B @ 9216db5the measured tokenizer itself: qwen3-32b. byte-identical tokenizer file: qwen3-14b, qwen3-8b, qwen3-30b-a3b, qwen3-30b-a3b-instruct-2507, qwen3-235b-a22b-2507, qwen3-next-80b-a3b-instruct, qwen3-next-80b-a3b-thinking. same vocabulary, merges and pre-tokenizer; identical counts on every text: qwen3-coder, qwen3-coder-30b-a3b-instruct, qwen3-coder-next, qwen3-vl-235b-a22b-instruct, qwen3-vl-30b-a3b-instruct, qwen3-vl-30b-a3b-thinking, qwen3-vl-32b-instruct, qwen3-vl-32b-thinking, qwen3-vl-8b-instruct.

The repo and commit checked for every model, and the evidence for each match, are in the JSON download (modelMapping).

14 candidates left out, and why
  • glm-4.7 (GLM 5): tokenizer.json differs from zai-org/GLM-5.3 in model.
  • gemma-4-12b-it (Gemma 4): tokenizer.json differs from google/gemma-4-26B-A4B-it in model.
  • gemma-4-e2b-it (Gemma 4): tokenizer.json differs from google/gemma-4-26B-A4B-it in model.
  • llama-3.2-11b-vision-instruct (Llama 3): gated, tokenizer.json differs from the measured file, and no byte-identical ungated mirror was found.
  • llama-3.2-90b-vision-instruct (Llama 3): gated, tokenizer.json differs from the measured file, and no byte-identical ungated mirror was found.
  • phi-4-reasoning (Phi-4): tokenizer.json differs from microsoft/phi-4 in model.
  • phi-4-reasoning-plus (Phi-4): tokenizer.json differs from microsoft/phi-4 in model.
  • mistral-small-3.2-24b-instruct (Mistral Small 3.1): no tokenizer.json in the repo.
  • nemotron-3.5-super-vl (Nemotron 3): tokenizer.json differs from nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 in model.
  • gpt-5.1 (OpenAI o200k_base): tiktoken names no encoding for "gpt-5.1" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.
  • gpt-5.2 (OpenAI o200k_base): tiktoken names no encoding for "gpt-5.2" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.
  • gpt-5.4 (OpenAI o200k_base): tiktoken names no encoding for "gpt-5.4" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.
  • gpt-5.5 (OpenAI o200k_base): tiktoken names no encoding for "gpt-5.5" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.
  • gpt-5.6-sol (OpenAI o200k_base): tiktoken names no encoding for "gpt-5.6-sol" except through the bare "gpt-5" prefix, which its own comment says can match non-existent models.

Method

  • Token counts: the 16-tokenizer study at /data/tokenizer-efficiency/ (version 4.0, measured 2026-11-15), each tokenizer's official file pinned to a commit. For each language, its texts are pooled (the 2,009 FLORES-200 sentences plus Wikipedia leads where we have them; English is Wikipedia and Project Gutenberg prose), then tokens ÷ characters and tokens ÷ words.
  • Model mapping: a priced model is linked to a measured tokenizer only when a script confirms it in the model's own Hugging Face repo: the same tokenizer file (same Git object ID), or the same vocabulary, merges, normalizer and pre-tokenizer with identical counts on all 52 texts, FLORES-200 sentences included. OpenAI's hosted models use the encoding OpenAI's tiktoken library names for them. Everything else is left out, including API-only models with no open weights.
  • Prices: standard-tier input and output per 1M tokens, read from each provider's own pricing page (the LLM API provider price database), and BroadNet's pricing.json. Batch, flex and priority tiers, free rows and cached-input prices are not used. Promotional prices are listed but not ranked.
  • Cost per 1M characters = tokens per character × price per 1M tokens. Cost per 1,000 words = tokens per word × 1,000 × price ÷ 1M.
  • Language tax: the extra cost of the same content in another language than in English, on the same model and provider, for content that is 1M words in English. It uses parallel sentences: FLORES-200 dev and devtest, 2,009 aligned sentences per language (CC BY-SA 4.0; we publish counts only), and for languages FLORES-200 lacks, Tatoeba sentence pairs (Gulf Arabic 49, Arabizi 39 sentences). Each extra-token figure has a 95% interval from a bootstrap over sentences (10,000 seeded resamples).
  • Limits: counts exclude chat-template and reasoning tokens; reasoning models bill thinking tokens as output. FLORES-200 is translated news and encyclopedia text, and Wikipedia leads are not your traffic. Treat the figures as a planning guide and measure your own text.

BroadNet rows: the four BroadNet chat models with a measured tokenizer, priced from pricing.json. BroadNet Gemma 4 VL 26B is BroadNet's build of Google's Gemma 4 26B; BroadNet has not published its tokenizer, so we assume it keeps Google's, and the row says so.

Version 1.1 (token data v4): the language tax now uses FLORES-200 parallel sentences (2,009 per language) instead of 33 to 73 Tatoeba sentences, adds Najdi and Mesopotamian Arabic, and gives a 95% interval for each extra-token figure. Gulf Arabic and Arabizi still use Tatoeba. Prices and the model mapping are unchanged in method.

Update frequency: costs are recomputed on every build from the latest provider prices and BroadNet's pricing.json; the model mapping is re-checked when the script is re-run; token counts change only when a tokenizer or the corpus changes.

What do the CSV columns mean?

One row per price row and language (9,270 rows).

ColumnMeaning
providerWho sells the model. "BroadNet Inference" rows are BroadNet, the owner of this site.
provider_idProvider ID, as in the LLM API provider price database.
modelModel name as the provider lists it.
model_slugModel ID in our price database; /llm-api-pricing/<model_slug>/ has every provider.
tokenizerMeasured tokenizer this model uses (see tokenizer_mapping).
tokenizer_mappingWhy the model maps to that tokenizer: measured, identical (byte-identical file), same-core (same vocabulary, merges and pre-tokenizer, identical counts), tiktoken (OpenAI's model table) or broadnet (BroadNet build of the measured model, assumed to keep its tokenizer).
languageLanguage or content kind: en, ar-msa, ar-gulf, ar-egyptian, ar-levantine, ar-najdi, ar-mesopotamian, arabizi, hi, ur, zh, fr, es, code, json.
input_usd_per_1m_tokensStandard-tier input price per 1M tokens, from the provider's page.
output_usd_per_1m_tokensStandard-tier output price per 1M tokens.
tokens_per_1k_charsMeasured tokens per 1,000 characters of this language with this tokenizer.
tokens_per_1k_wordsMeasured tokens per 1,000 words (Intl.Segmenter words; for Chinese, segmented words).
input_usd_per_1m_charsCost to send 1M characters of this language: tokens per character × input price.
output_usd_per_1m_charsCost to receive 1M characters of this language as output.
input_usd_per_1k_wordsCost to send 1,000 words.
output_usd_per_1k_wordsCost to receive 1,000 words.
promoyes if the provider marks the price as promotional; such rows are not ranked in the findings.
source_urlWhere the price was read.
checkedDate the price was checked.

How to cite this dataset

Our compilation and measurements (token counts, the model-to-tokenizer mapping, cost per character and per word, findings, tables and method): CC BY 4.0. Free to use, share and adapt with attribution and a link to this page.

Each price is a fact published by the provider named in its row, at the source URL given per row; we do not license those pages to you. Credit the provider as the source of each price. The texts behind the token counts keep their own licences (see the tokenizer efficiency dataset); FLORES-200 sentences (CC BY-SA 4.0) are not included anywhere in our files, only our counts.

Plain text

BroadRouter (2026). Real cost by language: what the same text costs on each LLM model and provider (version 1.1) [Data set]. BroadNet Technologies. https://broadrouter.com/data/real-cost-by-language/

BibTeX

@misc{broadrouter_real_cost_by_language,
  author       = {BroadRouter},
  title        = {Real cost by language: what the same text costs on each LLM model and provider},
  year         = {2026},
  version      = {1.1},
  publisher    = {BroadNet Technologies},
  howpublished = {\url{https://broadrouter.com/data/real-cost-by-language/}},
  note         = {Data set, data as of 2026-11-15},
}

Questions

How much does it cost to process Arabic text with an LLM API?

Sending 1M characters of Modern Standard Arabic costs $0.0077 to $6.39 across the 618 price rows here; the cheapest is Llama 3.1 8B Instruct on DeepInfra. Output costs more: the cheapest is Llama 3.1 8B Instruct on DeepInfra at $0.0155 per 1M characters.

Why does the same text cost more in Arabic or Hindi?

APIs bill per token, and a tokenizer trained on less Arabic or Hindi text splits those words into more pieces. The same sentence can need twice the tokens of its English version on one tokenizer and barely more on another, so the language changes which model is cheapest.

Which LLM API is cheapest for Hindi?

For input, OpenAI GPT OSS 120B on CoreWeave Forge at $0.0097 per 1M characters of Hindi; for output, Llama 3.1 8B Instruct on DeepInfra at $0.0209. These are list prices for the standard tier; check quality on your own text.

What is the language tax?

The extra you pay to process the same meaning in another language than in English, because it needs more tokens. Here it is measured on the same 2,009 FLORES-200 sentences in each language: for Modern Standard Arabic, 25% to 203% more tokens depending on the tokenizer.

Is the model with the lowest price per token the cheapest for every language?

No. Priced per character of Arabic, 150 pairs of models swap order; for Hindi, 463. Compare cost per character or per word in your language, not price per token.

Can I use this data?

Yes. Our compilation and measurements (token counts, the model-to-tokenizer mapping, cost per character and per word, findings, tables and method): CC BY 4.0. Each price is a fact published by the provider named in its row, at the source URL given per row; we do not license those pages to you. Credit the provider as the source of each price. The texts behind the token counts keep their own licences (see the tokenizer efficiency dataset); FLORES-200 sentences (CC BY-SA 4.0) are not included anywhere in our files, only our counts.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity