---
title: "Arabic WER explained - word error rate for Arabic ASR"
description: "What word error rate means, how to compute it, and why Arabic WER is so sensitive to diacritics, spelling normalisation and dialects. With Python and real numbers."
url: https://broadrouter.com/blog/arabic-asr-wer-explained/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Arabic ASR word error rate (WER) explained

By the BroadNet Inference team · 5 min read · Published 2026-11-14

Word error rate (WER) is the number of substituted, deleted and inserted words divided by the words in the reference transcript. Lower is better. Arabic WER is hard to compare because diacritics, alef and ta marbuta spelling and dialects inflate it. BroadNet reports 3.8 WER on FLEURS-ar and 23.4 on dialect speech for BroadNet Transcribe Arabic.

## Key takeaways

-   WER = (substitutions + deletions + insertions) / words in the reference. It can exceed 100%.
-   Arabic spelling variants (أ/إ/ا, ة/ه, ى/ي, diacritics) can count as errors unless you normalise both texts first.
-   Always state the normalisation you used. Two WER numbers with different normalisation are not comparable.
-   BroadNet reports 3.8 on FLEURS-ar, 21.1 on MASC-ar and 23.4 on its dialect set for BroadNet Transcribe Arabic.
-   Read the errors, not only the score: one wrong digit in an amount matters more than ten wrong particles.

**On this page**

1.  What is word error rate?
2.  Why is Arabic WER hard to compare?
3.  How much does normalisation change the score?
4.  How do you compute Arabic WER in Python?
5.  What WER does BroadNet report for Arabic?
6.  How do you measure WER on your own audio?
7.  What should you measure besides WER?

[Word error rate](https://broadrouter.com/glossary/word-error-rate/) (WER) is the number of substituted, deleted and inserted words divided by the number of words in the correct transcript. Lower is better. Arabic WER is easy to inflate: the same correct transcript can be written several ways, and a naive comparison counts every spelling choice, diacritic and dialect spelling as a mistake.

This post explains what WER measures, why Arabic makes it slippery and how to compute it so your numbers mean something. To build the test set itself, see [Arabic dialect speech recognition](https://broadrouter.com/blog/arabic-dialect-speech-recognition/); to choose a provider, see the [Arabic speech-to-text API buyer’s guide](https://broadrouter.com/blog/arabic-speech-to-text-api-buyers-guide/).

## What is word error rate?

Word error rate compares a machine transcript (the hypothesis) with a correct human transcript (the reference). It finds the minimum number of word edits that turn one into the other:

-   **S, substitutions:** a word replaced by a different word.
-   **D, deletions:** a reference word missing from the hypothesis.
-   **I, insertions:** an extra word in the hypothesis.

Then:

**WER = (S + D + I) / N**, where N is the number of words in the reference.

A WER of 0.10 (often written as 10) means about one error per ten reference words. Lower is better. Because insertions count, WER can go above 100% for a transcript full of invented words.

Character error rate (CER) is the same formula over characters. It is useful for Arabic, where a single wrong letter in a prefix makes the whole word count as wrong.

## Why is Arabic WER hard to compare?

### 1\. Diacritics

Arabic short vowels are written as optional marks (tashkeel) above and below letters: fatha, damma, kasra, sukun, shadda and tanween. Everyday text leaves them out. If your reference has “مَدْرَسَةٌ” and the model writes “مدرسة”, a strict comparison counts a substitution, even though any Arabic reader sees the same word.

### 2\. Spelling variants

Several letters have common, accepted variants:

| Variation | Example | Typical normalisation |
| --- | --- | --- |
| Hamza on alef | أحمد / احمد, إلى / الى | Map أ إ آ to ا |
| Ta marbuta vs ha | مدرسة / مدرسه | Map ة to ه |
| Alef maqsura vs ya | على / علي (in loose writing) | Map ى to ي |
| Tanween with alef | صباحاً / صباحا | Strip the diacritic |
| Tatweel (stretching) | جـمـيـل / جميل | Remove ـ |

### 3\. Clitics and word boundaries

Arabic attaches conjunctions, prepositions and pronouns to words: “وبمدرستهم” is “and in their school” in one written word. If the model writes “و بمدرستهم” with a space, the word count changes and WER punishes it twice. Some evaluations tokenise words into morphemes first; most do not. Know which you are reading.

### 4\. Dialects

There is no standard spelling for dialect speech, unlike [Modern Standard Arabic](https://broadrouter.com/glossary/msa/). An Egyptian speaker saying “I want” might be transcribed “عايز” or “عاوز”; a Gulf speaker’s “what” might be “وش” or “ويش”. Some transcribers also “correct” dialect into MSA. Each of these is a WER error even when the meaning is perfect. That is a large part of why dialect WER is so much higher than read-speech WER.

### 5\. Numbers and code-switching

Speakers often switch into English mid-sentence ([code-switching](https://broadrouter.com/glossary/code-switching/)). Is it “٢٥”, “25” or “خمسة وعشرين”? Is an English brand name written in Latin script or in Arabic letters? Decide a convention, apply it to the reference, and normalise the hypothesis to it.

## How much does normalisation change the score?

Reference: **ذهبت إلى المدرسة صباحاً** (“I went to school in the morning”)

Hypothesis: **ذهبت الى المدرسه صباحا**

Without normalisation, three of four words differ (إلى/الى, المدرسة/المدرسه, صباحاً/صباحا): **WER 75%.** After normalising both texts, they are identical: **WER 0%.** Same transcript, same quality, and the number moved from terrible to perfect. This is why a WER figure without its normalisation rules tells you little.

## How do you compute Arabic WER in Python?

```python
import re
import jiwer  # pip install jiwer

DIACRITICS = re.compile(r"[ً-ْٰ]")   # tanween, harakat, shadda, sukun, dagger alef
PUNCT = re.compile(r"[^\w\s]")

def normalise_ar(text: str) -> str:
    text = DIACRITICS.sub("", text)
    text = text.replace("ـ", "")                # tatweel
    text = re.sub("[أإآٱ]", "ا", text)
    text = text.replace("ى", "ي").replace("ة", "ه")
    text = text.translate(str.maketrans("٠١٢٣٤٥٦٧٨٩", "0123456789"))
    text = PUNCT.sub(" ", text)
    return " ".join(text.split())

refs = ["ذهبت إلى المدرسة صباحاً"]
hyps = ["ذهبت الى المدرسه صباحا"]

print("raw WER:", jiwer.wer(refs, hyps))
print("normalised WER:", jiwer.wer([normalise_ar(r) for r in refs], [normalise_ar(h) for h in hyps]))
print("normalised CER:", jiwer.cer([normalise_ar(r) for r in refs], [normalise_ar(h) for h in hyps]))
```

Apply exactly the same function to references and hypotheses, for every model you compare. Publish it next to your numbers.

## What WER does BroadNet report for Arabic?

BroadNet publishes lab measurements for [BroadNet Transcribe Arabic](https://broadrouter.com/models/cohere-transcribe-arabic/) in its [model catalogue](https://www.broadnet.ai/inference/models):

| Test set | What it is | WER reported by BroadNet |
| --- | --- | --- |
| FLEURS-ar | Clean read sentences | 3.8 |
| MASC-ar | Mixed real-world Arabic speech | 21.1 |
| BroadNet dialect set | Dialect speech | 23.4 |

BroadNet also reports the model at #2 of 37 (its September 2026 figure; the live board listed it #4 of 39 on 15 November 2026) on the [Open Universal Arabic ASR Leaderboard](https://huggingface.co/spaces/elmresearchcenter/open_universal_arabic_asr_leaderboard), and #1 among models whose licences permit serving through an API. These are BroadNet’s figures, not independent audits. The leaderboard applies one evaluation setup to every model, which makes its ranks more comparable than numbers from separate vendor pages.

Read the spread, not just the best number. The jump from 3.8 to 23.4 is the gap between read speech and spontaneous dialect for one model, so test every candidate on both kinds of audio. Your own audio, whether call-centre lines, WhatsApp voice notes or meetings, will land somewhere in that range, and only a test tells you where.

## How do you measure WER on your own audio?

1.  **Sample by condition.** Group clips by dialect and channel (phone, voice note, studio). The [dialect test-set method](https://broadrouter.com/blog/arabic-dialect-speech-recognition/) covers sample sizes and spelling conventions.
2.  **Transcribe with each candidate.** For BroadNet, send `language=ar`; without it, Arabic can come back transliterated into Latin letters. The [Arabic speech-to-text guide](https://broadrouter.com/guides/arabic-speech-to-text-api/) has the code, and the $5 starting credit covers about 1,190 minutes of audio.
3.  **Normalise and score.** Use the function above, report WER and CER per group.
4.  **Read the errors.** Sort substitutions by frequency. If the top errors are names, amounts or product codes, the business impact is bigger than the score suggests.

## What should you measure besides WER?

WER treats every word equally, but your product does not. A support bot cares about intent; an invoicing workflow cares about numbers; a compliance team cares about specific phrases. Add a task metric next to WER: intent accuracy after an LLM reads the transcript, or exact match on extracted amounts. A model with slightly higher WER can still win if its errors fall on words that do not matter.

For the cost side, put your monthly audio hours into the free [speech-to-text cost calculator](https://broadrouter.com/tools/speech-to-text-cost-calculator/), and see how the Arabic and 14-language models differ in [BroadNet Transcribe vs Transcribe Arabic](https://broadrouter.com/compare/cohere-transcribe-vs-cohere-transcribe-arabic/).

Free tools for this

-   [Arabic token counter](https://broadrouter.com/tools/arabic-token-counter/) Arabic vs English token counts per tokenizer: the "Arabic tax" in tokens and money.
-   [Arabic text normalizer](https://broadrouter.com/tools/arabic-text-normalizer/) Tashkeel, tatweel, alef, ya, ta marbuta, digits and invisible characters, with a matching Python function.
-   [Speech-to-text cost calculator](https://broadrouter.com/tools/speech-to-text-cost-calculator/) Hours, calls or voice notes to a monthly transcription bill, against your current provider.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Word error rate](https://broadrouter.com/glossary/word-error-rate/)

Word error rate (WER) is the standard accuracy measure for speech recognition: the number of word substitutions, deletions and insertions needed to turn a transcript into the correct reference, divided by the number of words in the reference. Lower is better.

[Speech-to-text](https://broadrouter.com/glossary/speech-to-text/)

Speech-to-text, also called automatic speech recognition (ASR), is technology that converts spoken audio into written text. A speech-to-text API accepts an audio file or stream and returns a transcript, usually priced per minute of audio. Accuracy is measured with word error rate and varies with accent, dialect and recording quality.

[Character error rate (CER)](https://broadrouter.com/glossary/character-error-rate/)

Character error rate (CER) measures transcription or OCR accuracy at the character level: the substitutions, deletions and insertions needed to turn the output into the reference text, divided by the number of characters in the reference. Lower is better, and 0 is a perfect match.

[Code-switching](https://broadrouter.com/glossary/code-switching/)

Code-switching is the use of two or more languages, or varieties of one language, within the same conversation or sentence, in speech or writing. Examples include Arabic speakers mixing English words into dialect, or moving between Modern Standard Arabic and their dialect.

[Arabic diacritics (tashkeel)](https://broadrouter.com/glossary/arabic-diacritics/)

Arabic diacritics, or tashkeel, are small marks written above or below letters to show short vowels, doubled consonants and the absence of a vowel. They are optional: most Arabic text, including news, omits them, so readers and software infer the vowels from context.

[Modern Standard Arabic (MSA)](https://broadrouter.com/glossary/msa/)

Modern Standard Arabic (MSA) is the standardized form of Arabic used in writing, news, education and formal speech across the Arab world. It is nobody's native everyday language: people speak regional dialects such as Egyptian or Gulf Arabic and switch to MSA in formal settings.

[Arabic text normalization](https://broadrouter.com/glossary/arabic-text-normalization/)

Arabic text normalization is the step that maps spelling variants of the same word to one form before matching: removing diacritics and tatweel, unifying alef forms, and often mapping ى to ي and ة to ه. It is applied to search keys and evaluation, not to text you display.

[Dialectal Arabic](https://broadrouter.com/glossary/dialectal-arabic/)

Dialectal Arabic is the everyday spoken Arabic of a region or city, such as Egyptian, Gulf, Levantine or Maghrebi Arabic. Dialects differ from Modern Standard Arabic in sounds, words and grammar, have no standard spelling, and are increasingly written online, which makes them harder for language models.

## Questions

### What is a good Arabic WER?

On clean read speech, single digits are achievable; BroadNet reports 3.8 on FLEURS-ar. On real-world and dialect audio, expect much higher numbers; BroadNet reports 21.1 on MASC-ar and 23.4 on its dialect set. Compare models on your own audio with the same normalisation.

### Should I strip diacritics before computing Arabic WER?

Usually yes. Most Arabic is written without short-vowel marks, and models add them inconsistently. Strip them from both reference and hypothesis unless your product needs vowelled text.

### What is CER and when should I use it?

Character error rate is the same calculation over characters instead of words. It is less harsh on Arabic, where one wrong prefix letter makes a whole word wrong, and is useful alongside WER.

### Can WER be over 100%?

Yes. Insertions count as errors, so a model that invents many extra words can make more errors than there are words in the reference.

### How do I calculate WER for Arabic in Python?

Install jiwer, normalise both the reference and the model output with the same function (strip diacritics and tatweel, unify alef, yaa and ta marbuta, convert Eastern Arabic digits), then call jiwer.wer(references, hypotheses). Report the normalisation next to the score.

### Why is Arabic WER higher than English WER?

Partly real difficulty (dialects, clitics, code-switching) and partly scoring: Arabic has accepted spelling variants and optional diacritics, and a strict comparison counts each one as an error. Normalising both texts removes the scoring part.

## Keep reading

-   [Arabic speech-to-text API: how to choose one in 2026](https://broadrouter.com/blog/arabic-speech-to-text-api-buyers-guide/)
-   [How to transcribe call center recordings: an Arabic-ready pipeline](https://broadrouter.com/blog/transcribe-call-center-recordings/)
-   [Arabic RAG: normalisation, chunking and retrieval that works](https://broadrouter.com/blog/arabic-rag-search/)
-   [Multilingual speech-to-text API: 14 languages, one endpoint](https://broadrouter.com/blog/multilingual-speech-to-text-api/)
-   [Add diacritics to Arabic text with an LLM: prompt, checks, DER](https://broadrouter.com/blog/arabic-diacritization-llm/)
-   [Arabic grammar and spelling correction with an LLM: a how-to](https://broadrouter.com/blog/arabic-grammar-spelling-correction-llm/)
-   [Guide: Arabic speech-to-text API, step by step](https://broadrouter.com/guides/arabic-speech-to-text-api/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-arabic-asr-wer-explained&utm_content=page-cta) ·

**Sources**

-   [BroadNet model catalogue, speech to text: WER and leaderboard rank as reported by BroadNet (fetched 2026-11-14)](https://www.broadnet.ai/inference/models)
-   [Open Universal Arabic ASR Leaderboard, Hugging Face Space (fetched 2026-11-14)](https://huggingface.co/spaces/elmresearchcenter/open_universal_arabic_asr_leaderboard)
-   [Wang, Alhmoud and Alqurishi, Open Universal Arabic ASR Leaderboard (Interspeech 2025; fetched 2026-11-14)](https://www.isca-archive.org/interspeech_2025/wang25_interspeech.html)
-   [Conneau et al., FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech (arXiv; fetched 2026-11-14)](https://arxiv.org/abs/2205.12446)
-   [jiwer, a Python package for WER and CER (fetched 2026-11-14)](https://github.com/jitsi/jiwer)
