---
title: "Arabic text summarization with an LLM - how to do it"
description: "Summarise Arabic articles, reports and dialect messages with an LLM API: prompt patterns, word-count control, number checks, and why default ROUGE gives Arabic 0."
url: https://broadrouter.com/blog/arabic-text-summarization-llm/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Arabic text summarization with an LLM: prompts, length and scoring

By the BroadNet Inference team · 7 min read · Published 2026-11-15 · Updated 2026-11-16

Name the register (usually Modern Standard Arabic, even for dialect input) and a word range in the prompt. Then check in code: count the words, confirm every number in the summary appears in the source, and retry once with the problem stated. Score with an Arabic-aware ROUGE tokenizer plus human review; default rouge-score gives Arabic 0.

## Key takeaways

-   Google's rouge-score 0.1.2 strips non-Latin characters by default. We ran it on two Arabic sentences and got an F1 of 0, whatever the overlap.
-   The same pair scored 0.6 with a whitespace Arabic tokenizer and 0.8 with crude prefix stripping. Never compare ROUGE numbers produced with different tokenizers.
-   Ask for a word range, then count. Retry once with the measured length in the message; send it to review if it still misses.
-   Check that every number in the summary is in the source. It is a cheap test for one common kind of invented detail.
-   Public Arabic summarization data is mostly MSA news: XL-Sum (46,897 Arabic pairs), EASC (153 articles) and LANS. Dialect data is scarce, so build your own test set.

**On this page**

1.  What should an Arabic summarization prompt say?
2.  How do you summarise dialect text?
3.  How do you control summary length?
4.  How do you catch invented details?
5.  How do you summarise long Arabic documents?
6.  How do you evaluate Arabic summaries?
7.  Which BroadNet model should you use, and what does it cost?
8.  Where BroadNet Inference fits

Summarising Arabic text with an LLM takes one request. Getting summaries you can ship takes three decisions the request doesn’t make for you: which register to write in, how long the summary must be, and how you will know it is right. This post covers written text: articles, reports, tickets and long message threads. For spoken meetings, the [Arabic meeting minutes pipeline](https://broadrouter.com/blog/arabic-meeting-minutes-from-recordings/) adds the transcription step and section-by-section merging.

## What should an Arabic summarization prompt say?

Five lines carry most of the result:

1.  **Language and register.** “Modern Standard Arabic” (الفصحى), or a named dialect. Without it, a model summarising Egyptian or Gulf input may answer in a mix.
2.  **A word range.** “40 to 60 words”. Ranges are easier to hit than exact counts.
3.  **Facts as written.** Numbers, dates and names copied from the text, not rounded or converted.
4.  **Nothing new.** No background the text does not contain.
5.  **Shape.** One paragraph, or three bullet points, or JSON with fixed fields.

Instructions can be in Arabic or English; either way, state the output language and register explicitly. [Arabic prompt engineering](https://broadrouter.com/blog/arabic-prompt-engineering/) covers register, numerals and mixed-script text in more depth. Here is the prompt used in the code below:

```text
لخّص النص التالي بالعربية الفصحى في {lo} إلى {hi} كلمة.
- اذكر الأرقام والتواريخ والأسماء كما وردت في النص فقط.
- لا تضف معلومات غير موجودة في النص.
- اكتب فقرة واحدة دون عناوين أو نقاط.
```

## How do you summarise dialect text?

Most business text in Arabic is not MSA: customer reviews, WhatsApp threads and social posts are in dialect, often mixed with English or [Arabizi](https://broadrouter.com/blog/arabizi-llm-handling/). Three choices help:

-   **Summarise into MSA by default.** It is neutral across regions and easier to search. If the summary goes back to the same customer, match their dialect instead.
-   **Keep key quotes verbatim.** Ask for one short quote in the original wording when tone matters, such as a complaint. Paraphrasing dialect into MSA may soften it.
-   **Test on your dialects.** Dialect understanding varies between models; see [how LLMs cope with Gulf and Egyptian Arabic](https://broadrouter.com/blog/arabic-llm-gulf-egyptian-dialects/).

Public data won’t do that testing for you. The large Arabic summarization sets are MSA news, and we did not find a public Gulf or Egyptian summarization benchmark to point you to.

## How do you control summary length?

Ask in words, then measure in words. Don’t trust the model’s own count, and don’t use a token budget as the unit: tokens per Arabic word vary by model. We measured a 96-word Arabic news paragraph at 166 tokens with the official Qwen3.8 27B tokenizer, and its 58-word summary at 114 tokens. Across the tokenizers of BroadNet’s four chat models, counted with the [Arabic token counter](https://broadrouter.com/tools/arabic-token-counter/)’s tokenizer files, the same paragraph ranged from 166 to 224 tokens. [Tokens per word in Arabic](https://broadrouter.com/blog/tokens-per-word-arabic-english/) has the larger measurement.

Splitting on whitespace is the right count for most uses, because it is what readers see. Arabic attaches و, ب, ل and ال to the word, so a 50-word Arabic summary carries more than 50 English words of content. Set your range from Arabic examples you like, not from an English target.

Two more rules on length:

-   **`max_tokens` is not a length control.** It cuts the reply off mid-sentence. Set it well above the summary size, especially on reasoning models: BroadNet’s docs say DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before they answer, and that reasoning counts toward `max_tokens`. See [empty responses from reasoning models](https://broadrouter.com/guides/reasoning-tokens-empty-response/).
-   **Retry once, with the number.** “Your summary is 17 words; I asked for 40 to 60” fixes more than repeating the instruction.

## How do you catch invented details?

Abstractive summarizers invent things. Maynez et al. (ACL 2020) found “substantial amounts of hallucinated content” in all the model summaries their annotators reviewed; that was before today’s LLMs, but no model is immune. You can’t check meaning cheaply in code, but you can check numbers: every number in the summary should appear in the source. Map Eastern Arabic digits (٠–٩) and Persian digits (۰–۹) to 0–9 first, or ١٢ and 12 will look different.

```python
import os, re
from openai import OpenAI

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])

PROMPT = """لخّص النص التالي بالعربية الفصحى في {lo} إلى {hi} كلمة.
- اذكر الأرقام والتواريخ والأسماء كما وردت في النص فقط.
- لا تضف معلومات غير موجودة في النص.
- اكتب فقرة واحدة دون عناوين أو نقاط.

النص:
{text}"""

DIGITS = str.maketrans("٠١٢٣٤٥٦٧٨٩۰۱۲۳۴۵۶۷۸۹", "01234567890123456789")
def numbers(s: str) -> set[str]:
    return set(re.findall(r"\d+(?:[.,٫]\d+)?", s.translate(DIGITS)))

def words(s: str) -> int:
    return len(s.split())          # clitics (و، ال، ب) stay attached: count what readers see

def check(source: str, summary: str, lo: int, hi: int) -> list[str]:
    issues = []
    n = words(summary)
    if not lo <= n <= hi:
        issues.append(f"الطول {n} كلمة، والمطلوب من {lo} إلى {hi}.")
    extra = numbers(summary) - numbers(source)
    if extra:
        issues.append("أرقام غير موجودة في النص: " + "، ".join(sorted(extra)))
    return issues

def summarize(text: str, lo=40, hi=60, model="broadnet/qwen3.8-27b", attempts=2) -> dict:
    messages = [{"role": "user", "content": PROMPT.format(lo=lo, hi=hi, text=text)}]
    for i in range(attempts):
        r = client.chat.completions.create(model=model, messages=messages, max_tokens=2000)
        summary = (r.choices[0].message.content or "").strip()
        issues = check(text, summary, lo, hi)
        if not issues:
            return {"summary": summary, "attempts": i + 1, "issues": []}
        messages += [{"role": "assistant", "content": summary},
                     {"role": "user", "content": "صحّح الملخص: " + " ".join(issues)}]
    return {"summary": summary, "attempts": attempts, "issues": issues}   # send to review
```

We tested this against a local mock server (no live API call) with a synthetic 96-word article about a city bus plan. The mock’s first reply was 17 words and said ٣٠٠ buses where the article said ٢٤٠. `check` returned both problems: the measured length, and “numbers not in the text: 300”. The second, corrected reply passed at 58 words, so the function returned it with `attempts: 2`.

The number check misses things: a wrong name, a reversed decision, a claim with no number in it. It is a filter, not a verdict. For names, reuse the span check from [Arabic named entity extraction](https://broadrouter.com/blog/arabic-named-entity-extraction-llm/): extract entities from the summary and confirm each one appears in the source.

## How do you summarise long Arabic documents?

If the text fits the context window with room to spare, send it whole. [Qwen3.8 27B](https://broadrouter.com/models/qwen3.8-27b/) takes 262,144 tokens per request and [GLM 5.3](https://broadrouter.com/models/glm-5.3/) 1,048,576, prompt and answer together; [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/) takes 32,768. Long inputs have a known weakness, though: models use facts in the middle of a long context less reliably ([lost in the middle](https://broadrouter.com/glossary/lost-in-the-middle/)). For a long report, summarise each section, then summarise the summaries, and keep section headings in the final prompt. The meeting post shows the [merge step](https://broadrouter.com/blog/arabic-meeting-minutes-from-recordings/) in code; [long context vs RAG](https://broadrouter.com/blog/long-context-vs-rag/) covers when to retrieve instead.

## How do you evaluate Arabic summaries?

Use three layers: an automatic overlap score to catch regressions, a semantic score for a second opinion, and people for the decision.

**ROUGE, with the right tokenizer.** Google’s rouge-score package (version 0.1.2, still the current release on PyPI on 16 November 2026) lowercases text and replaces every character outside a–z and 0–9 with a space. Arabic disappears. We checked:

```python
import re
from rouge_score import rouge_scorer

ref = "أعلنت البلدية خطة لتوسعة النقل العام تضيف ١٢ خطاً للحافلات"
hyp = "البلدية تعلن خطة توسعة للنقل العام بإضافة ١٢ خطا للحافلات"

print(rouge_scorer.RougeScorer(["rouge1"]).score(ref, hyp)["rouge1"].fmeasure)   # 0.0

class ArabicTokenizer:
    """Whitespace tokens, no diacritics, unified alef, optional light prefix stripping."""
    def __init__(self, strip_prefixes=False):
        self.strip = strip_prefixes
    def tokenize(self, text):
        text = re.sub(r"[ً-ٰٟـ]", "", text)
        text = re.sub("[إأآ]", "ا", text).replace("ى", "ي").replace("ة", "ه")
        toks = re.findall(r"\w+", text)
        if self.strip:   # crude: drop و/ب/ل/ال prefixes on longer words
            toks = [re.sub(r"^(وال|بال|لل|ال|و|ب|ل)(?=\w{3,})", "", t) for t in toks]
        return toks

for strip in (False, True):
    s = rouge_scorer.RougeScorer(["rouge1"], tokenizer=ArabicTokenizer(strip))
    print(strip, round(s.score(ref, hyp)["rouge1"].fmeasure, 3))   # False 0.6, True 0.8
```

The same two sentences score 0, 0.6 or 0.8 depending only on tokenization. That is the real lesson: Arabic morphology (للنقل against النقل, بإضافة against تضيف) moves ROUGE a lot, so a score means nothing without its tokenizer. The XL-Sum authors publish a multilingual ROUGE fork with language-specific stemming, including Arabic; use it, or your own tokenizer, and keep it fixed across every run you compare.

**BERTScore** compares meaning through embeddings, so it tolerates paraphrase. For languages other than English and Chinese its default model is bert-base-multilingual-cased. Treat it as a second signal, not a replacement for reading.

**People, with a rubric.** Have an Arabic reader score 50 to 100 summaries on three questions: is anything important missing, is anything stated that the source doesn’t say, and is the register right. An [LLM judge](https://broadrouter.com/glossary/llm-as-judge/) can pre-screen at volume once its scores agree with your readers on a sample. [Evaluating an LLM before switching](https://broadrouter.com/blog/evaluate-llm-before-switching/) covers building that set.

For public reference data:

| Dataset | What it holds | Notes |
| --- | --- | --- |
| [XL-Sum](https://huggingface.co/datasets/csebuetnlp/xlsum) Arabic | 46,897 BBC Arabic article and summary pairs (37,519 / 4,689 / 4,689) | CC BY-NC-SA 4.0: non-commercial research only |
| [EASC](https://huggingface.co/datasets/drelhaj/EASC) | 153 articles, five human extractive summaries each | Small; Wikipedia and two newspapers |
| [LANS](https://arxiv.org/abs/2210.13600) | 8.4 million articles with summaries from 22 Arab newspapers, 1999 to 2019 | Summaries come from site metadata |

All three are MSA news. Your support tickets and reports are not, so a sample of your own text, summarised by a person, is the test that counts.

## Which BroadNet model should you use, and what does it cost?

For a pilot, [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/) runs on the $5 starting credit at $0.10 input and $0.22 output per 1M tokens; its 32,768-token window fits most single articles. [Qwen3.8 27B](https://broadrouter.com/models/qwen3.8-27b/), which BroadNet reports testing for fluent Arabic writing and Gulf and Egyptian dialect understanding, costs $0.175 input and $2.17 output after a top-up. Use GLM 5.3 when one document is longer than Qwen’s window.

The arithmetic for 10,000 articles of 1,000 tokens each, with 150-token summaries: 10 million input tokens and 1.5 million output tokens, so on Qwen3.8 27B that is 10 × $0.175 plus 1.5 × $2.17. Add reasoning tokens, which bill as output, and any retries; measure both on a sample from the usage block. If the prompt is the same for every article, put it first so it can be billed at the [cached input](https://broadrouter.com/glossary/cached-input-tokens/) rate.

## Where BroadNet Inference fits

BroadRouter is BroadNet’s own site. BroadNet Inference serves the summarization step through an OpenAI-compatible API, so the code above runs with the [OpenAI Python SDK](https://broadrouter.com/guides/openai-python-sdk/). Its docs say prompts and answers are not retained after the request. It publishes no Arabic summarization benchmark of its own, and nothing here measures summary quality on its models; run the checks above on your own texts. For spoken input, pair it with BroadNet Transcribe Arabic as in the [meeting minutes post](https://broadrouter.com/blog/arabic-meeting-minutes-from-recordings/).

Free tools for this

-   [Arabic token counter](https://broadrouter.com/tools/arabic-token-counter/) Arabic vs English token counts per tokenizer: the "Arabic tax" in tokens and money.
-   [Arabic text normalizer](https://broadrouter.com/tools/arabic-text-normalizer/) Tashkeel, tatweel, alef, ya, ta marbuta, digits and invisible characters, with a matching Python function.
-   [Speech-to-text cost calculator](https://broadrouter.com/tools/speech-to-text-cost-calculator/) Hours, calls or voice notes to a monthly transcription bill, against your current provider.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Tokenizer](https://broadrouter.com/glossary/tokenizer/)

A tokenizer is the component that converts text into the sequence of integer token IDs a language model reads, and converts generated IDs back into text. Each model ships its own tokenizer, so the same text can produce a different number of tokens on different models.

[Input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/)

Input tokens are the tokens a language model reads in a request, including the system prompt, conversation history, tool definitions and images. Output tokens are the tokens it generates in reply, including any reasoning. LLM APIs usually price them separately, and output usually costs more.

[Modern Standard Arabic (MSA)](https://broadrouter.com/glossary/msa/)

Modern Standard Arabic (MSA) is the standardized form of Arabic used in writing, news, education and formal speech across the Arab world. It is nobody's native everyday language: people speak regional dialects such as Egyptian or Gulf Arabic and switch to MSA in formal settings.

[Arabizi](https://broadrouter.com/glossary/arabizi/)

Arabizi is Arabic written informally in Latin letters, using digits for Arabic sounds with no Latin equivalent, such as 3 for ع. It is common in chat and social media, is mostly dialectal, has no standard spelling, and is often mixed with English or French.

[Context window](https://broadrouter.com/glossary/context-window/)

A context window is the maximum number of tokens a language model can work with in a single request. It covers the prompt, conversation history, tool definitions, images and the generated output, including any reasoning tokens. Anything that does not fit must be cut, summarised or split across several requests.

[Dialectal Arabic](https://broadrouter.com/glossary/dialectal-arabic/)

Dialectal Arabic is the everyday spoken Arabic of a region or city, such as Egyptian, Gulf, Levantine or Maghrebi Arabic. Dialects differ from Modern Standard Arabic in sounds, words and grammar, have no standard spelling, and are increasingly written online, which makes them harder for language models.

[Embeddings](https://broadrouter.com/glossary/embeddings/)

Embeddings are lists of floating-point numbers (vectors) that a model produces to represent a piece of text, an image or other input. Inputs with similar meaning get vectors that are close together, so distance between vectors can rank search results, cluster documents or find related items.

## Questions

### Can an LLM summarise Arabic text well?

Yes, but fluent is not the same as accurate. Check lengths and numbers in code, and have an Arabic reader score a sample for missing points and invented details before you rely on it.

### Should a summary of dialect text be in dialect or MSA?

Usually MSA, because it reads as neutral and searchable across regions. Keep short quotes in the original dialect when the wording matters, such as a complaint. Say which you want in the prompt; models otherwise drift between the two.

### Why does ROUGE give 0 for my Arabic summaries?

The widely used rouge-score Python package replaces characters outside a-z and 0-9 with spaces by default, so Arabic text produces no tokens. Pass your own tokenizer, or use the multilingual ROUGE fork from the XL-Sum authors.

### How do I control the length of an Arabic summary?

Ask for a word range, not a token count, and measure the reply by splitting on whitespace. If it misses, resend once with the measured length. Remember that Arabic words often carry attached prefixes, so a 50-word Arabic summary holds more than 50 English words of meaning.

### Which dataset can I use to test Arabic summarization?

XL-Sum has 46,897 Arabic BBC article and summary pairs (non-commercial licence). EASC has 153 articles with five human extractive summaries each. Both are MSA news, so add a sample of your own texts, especially dialect ones.

## Keep reading

-   [Arabic meeting minutes from recordings: an API pipeline](https://broadrouter.com/blog/arabic-meeting-minutes-from-recordings/)
-   [Arabic prompt engineering: register, RTL, numerals and mixed text](https://broadrouter.com/blog/arabic-prompt-engineering/)
-   [Gulf, Egyptian and Levantine Arabic: how LLMs handle dialects](https://broadrouter.com/blog/arabic-llm-gulf-egyptian-dialects/)
-   [Choosing an LLM API for Arabic: what to test](https://broadrouter.com/blog/choosing-an-llm-for-arabic/)
-   [Tokens per word in Arabic vs English: why Arabic costs more](https://broadrouter.com/blog/tokens-per-word-arabic-english/)
-   [Long context vs RAG: when 1M tokens replaces a vector DB](https://broadrouter.com/blog/long-context-vs-rag/)
-   [Add diacritics to Arabic text with an LLM: prompt, checks, DER](https://broadrouter.com/blog/arabic-diacritization-llm/)
-   [Arabic grammar and spelling correction with an LLM: a how-to](https://broadrouter.com/blog/arabic-grammar-spelling-correction-llm/)
-   [Guide: Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)
-   [Guide: Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-arabic-text-summarization-llm&utm_content=page-cta) ·

**Sources**

-   [Hasan et al., XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages, Findings of ACL 2021 (checked 2026-11-15)](https://aclanthology.org/2021.findings-acl.413/)
-   [XL-Sum dataset card on Hugging Face: Arabic split 37,519 / 4,689 / 4,689, CC BY-NC-SA 4.0 (checked 2026-11-15)](https://huggingface.co/datasets/csebuetnlp/xlsum)
-   [XL-Sum repository, multilingual ROUGE scoring with language-specific stemming (checked 2026-11-15)](https://github.com/csebuetnlp/xl-sum)
-   [Google Research rouge-score tokenizer source: lowercases and replaces non-alphanumeric characters (checked 2026-11-15)](https://github.com/google-research/google-research/blob/master/rouge/tokenize.py)
-   [El-Haj, Kruschwitz and Fox, Essex Arabic Summaries Corpus (EASC), Hugging Face mirror (checked 2026-11-15)](https://huggingface.co/datasets/drelhaj/EASC)
-   [Alhamadani et al., LANS: Large-scale Arabic News Summarization Corpus, arXiv 2210.13600 (checked 2026-11-15)](https://arxiv.org/abs/2210.13600)
-   [Maynez et al., On Faithfulness and Factuality in Abstractive Summarization, ACL 2020 (checked 2026-11-15)](https://aclanthology.org/2020.acl-main.173/)
-   [BERTScore repository: default multilingual model for languages other than English and Chinese (checked 2026-11-15)](https://github.com/Tiiiger/bert_score)
-   [BroadNet Inference API reference and pricing: models, context lengths, reasoning and max\_tokens (checked 2026-11-15)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-arabic-text-summarization-llm&utm_content=sources)
