To make retrieval-augmented generation (RAG) work on Arabic, normalise documents and queries with the same spelling rules, chunk by paragraph, search with both an Arabic keyword analyzer and a multilingual embedding model, and have the LLM answer only from the retrieved passages with citations.
The pattern, from the original RAG paper, is retrieve, then generate. Most Arabic RAG failures happen in the first half. A user types “الفاتوره” and the index holds “الفاتورة”; a policy says “إلغاء” and the user writes “الغاء”; the embedding model was mostly trained on English. Fix retrieval and the LLM part tends to look after itself.
What does an Arabic RAG pipeline look like?
- Extract text from PDFs, web pages and help-centre articles. For scanned Arabic PDFs, see Arabic OCR vs vision LLMs.
- Normalise a copy of the text for indexing.
- Chunk by structure: headings and paragraphs.
- Index twice: a keyword index and a vector index.
- Retrieve with the normalised query from both, merge and rerank.
- Generate an answer from the top passages, citing them.
How should you normalise Arabic text for search?
Arabic allows several spellings of the same word, and users rarely type diacritics or hamza consistently. Normalise the indexed copy and every query with the same function:
| Rule | Characters | Why |
|---|---|---|
| Unify alef | أ إ آ ٱ → ا | Hamza is often omitted when typing |
| Alef maqsura | ى → ي | Common in Egyptian typing and loose spelling |
| Ta marbuta | ة → ه | Users type ه at word end |
| Strip diacritics | U+064B to U+0652, U+0670 | Most text and queries are unvowelled |
| Remove tatweel | ـ (U+0640) | Decorative stretching |
| Digits | ٠–٩ → 0–9 | Match numbers in either form |
Try each option on your own text with the Arabic text normalizer, which also writes a matching Python function.
import re
_DIAC = re.compile(r"[ً-ْٰ]")
def normalise(text: str) -> str:
text = _DIAC.sub("", text).replace("ـ", "")
text = re.sub("[أإآٱ]", "ا", text)
text = text.replace("ى", "ي").replace("ة", "ه")
return text.translate(str.maketrans("٠١٢٣٤٥٦٧٨٩", "0123456789"))
Code points are in the Unicode Arabic chart. Two cautions:
- Never show the normalised text. Store the original next to it and display, cite and send the original to the LLM. Normalised Arabic looks misspelled to readers.
- Some merges lose meaning. ة → ه merges a few word pairs, and ى → ي merges “على” (on) with “علي” (Ali). For search recall, that trade is usually worth it; test it on your own queries.
The same rules are central to measuring Arabic speech-recognition accuracy, so you can reuse one function across your stack.
How should you chunk Arabic documents?
- Split on structure. Headings, numbered articles in contracts (المادة ١، المادة ٢) and paragraph breaks are natural boundaries.
- A few hundred words per chunk, with a sentence or two of overlap. Arabic words carry more morphemes than English words, so a chunk covers more meaning per word than its length suggests.
- Sentence boundaries: split on . ؟ ! and the Arabic comma ، only as a last resort. Many Arabic texts use long sentences joined with و, so paragraph-level splitting is safer.
- Carry context. Prefix each chunk with the document title and section heading, in Arabic, so a chunk that says “تُطبَّق هذه الرسوم” (these fees apply) still says which fees.
The RAG chunking calculator turns chunk size, overlap and top-k into chunk counts, tokens per query and a monthly cost.
Why combine keyword search and embeddings?
Keyword search
Arabic morphology defeats naive keyword matching: كتاب, الكتاب, بالكتاب, وكتابه and كتب all relate to “book”. Use an analyzer that handles Arabic. Elasticsearch ships an Arabic language analyzer that chains Arabic stop words, digit normalisation, Arabic normalisation and an Arabic stemmer, so prefixed forms such as الكتاب and وكتاب match more often. Check what it merges on your own vocabulary before you rely on it.
Embeddings
Embedding models map text to vectors so that passages with similar meaning sit close together, even when they share no words. For Arabic, choose a multilingual model and test it:
- Prefer models trained on many languages, such as BGE M3, which its authors describe as supporting more than 100 working languages. English-only models give poor Arabic vectors.
- Test cross-language retrieval if users ask in English about Arabic documents, or the reverse.
- Embed the original or the normalised text, but be consistent between documents and queries. Test both; results vary by model.
BroadNet Inference does not offer an embeddings endpoint: its documented endpoints are chat completions and audio transcriptions. Use an embedding provider you trust, or host an open multilingual embedding model in a vector database, and use BroadNet for generation. The embedding cost calculator prices indexing and queries across providers.
Merge and rerank
Run both searches, merge results with reciprocal rank fusion (score each document by the sum of 1 / (k + rank) across lists), and keep the top 10 to 30 passages. A reranker or a quick LLM relevance check can trim that further, but with long-context models you can often skip it.
How do you generate cited answers?
Give the model the passages with ids and ask it to answer only from them:
import os
from openai import OpenAI
client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])
def answer(question: str, passages: list[dict]) -> str:
context = "\n\n".join(f"[{p['id']}] {p['title']}\n{p['text']}" for p in passages)
system = ("أجب عن سؤال المستخدم بالعربية الفصحى المبسطة، اعتماداً فقط على المقاطع المرفقة. "
"اذكر رقم المقطع بين قوسين مربعين بعد كل معلومة. "
"إذا لم تجد الإجابة في المقاطع فقل ذلك بوضوح.")
r = client.chat.completions.create(
model="broadnet/deepseek-v4.1-flash",
max_tokens=4000,
messages=[{"role": "system", "content": system},
{"role": "user", "content": f"المقاطع:\n{context}\n\nالسؤال: {question}"}],
)
return r.choices[0].message.content
The Arabic system prompt says: “Answer the user’s question in simple Modern Standard Arabic, based only on the attached passages. Cite the passage number in square brackets after each fact. If you cannot find the answer in the passages, say so clearly.” English instructions work just as well; what matters is the rule to answer only from the passages and to cite.
Put the passages before the question. If the same documents appear in many requests, keep them in the same order at the start of the prompt so cached input pricing applies; see cached input tokens.
Which model?
| Model | Context | Why |
|---|---|---|
| DeepSeek V4.1 Flash | 1,048,576 | Lowest cached-input rate of these three on BroadNet ($0.0055 per 1M tokens), room for many passages, flat plans available |
| GLM 5.3 | 1,048,576 | Tools and structured output for agentic retrieval |
| Qwen3.8 27B | 262,144 | BroadNet tested it in Arabic: fluent writing, Gulf and Egyptian dialects understood |
With a million-token context, you can retrieve generously, say 50 passages instead of 5, and let the model pick. That reduces the damage when retrieval ranks the right passage at number 12. For small document sets, you may not need retrieval at all; see long context vs RAG and check whether your corpus fits with the context window calculator. Prices for all models are on the pricing page.
How do you evaluate Arabic RAG?
- Write 50 to 100 real questions, in the mix of MSA, dialect and English your users actually type.
- Mark the passage that answers each one.
- Measure retrieval first: is the right passage in the top 10? Compare keyword only, embeddings only and hybrid, with and without normalisation.
- Then measure answers: correct, cited, and “not found” when the answer is missing.
- Include misspelled and dialect queries on purpose. Users write “وين الفاتوره” (Gulf/Levantine: where is the invoice), not “أين الفاتورة”.
Fix retrieval before you change the generation model: if the right passage never reaches the prompt, no model can cite it.

