WER asks how many words were wrong. CER asks how many letters were wrong, which is often the fairer question for Arabic.
How it works
CER uses the same edit-distance idea as word error rate, applied to characters. The Hugging Face metric card gives the formula:
CER = (S + D + I) / N, where S is substitutions, D deletions, I insertions and N the number of characters in the reference (N = S + D + C, with C the correct characters).
A worked Arabic example. The reference is “وبالكتاب” (“and with the book”, one word, 8 letters). The transcript is “وبلكتاب”, missing one alef:
| Metric | Errors | Reference units | Score |
|---|---|---|---|
| WER | 1 wrong word | 1 word | 1.0 (100%) |
| CER | 1 deleted letter | 8 characters | 0.125 (12.5%) |
One missing letter makes the whole word wrong for WER. CER shows the transcript was almost right.
In Python, the jiwer package computes it with jiwer.cer(reference, hypothesis), and jiwer.process_characters also returns the alignment. Normalise both sides first, as for WER: punctuation, spacing, and for Arabic, diacritics and alef or yaa forms. See Arabic text normalization.
Why it matters
- Arabic morphology. Arabic attaches conjunctions, prepositions and pronouns to words as clitics, so one word can carry what English writes as three or four. A single wrong prefix letter costs a full word in WER. CER weighs it as one character.
- Languages without spaces. WER needs word boundaries. The Hugging Face card says CER is useful “especially for multilingual datasets where WER is not suitable given the diversity of languages.”
- Error type. A model with high WER but low CER is getting words nearly right: spelling, hamza or clitic slips that normalisation or a correction step may fix. High CER means the model is missing whole words or content.
- OCR too. The same metric scores text read from images, so it is useful when you compare OCR pipelines.
The metric card also warns that CER does not tell you why errors happen. Read sample transcripts alongside the score.
On BroadNet Inference
BroadNet publishes WER, not CER, for BroadNet Transcribe Arabic and BroadNet Transcribe. To get CER, transcribe a test set of your own recordings and score it yourself. The Arabic WER explained post has a normalisation function and a jiwer script that prints both WER and CER, and the Arabic speech-to-text guide has the request code. Always send language=ar: without it, Arabic can come back transliterated into Latin letters, which wrecks both scores.
Common mistakes
- Counting spaces inconsistently. Decide whether spaces count as characters and apply the same rule to every model you compare.
- Comparing CER to someone else’s WER. They are different scales. Compare like with like.