Pull every person, company and place out of a contract or a news story, with a type for each. That is NER, and it is one of the most common jobs given to an LLM.
How it works
The classic benchmark is the CoNLL-2003 shared task. It used English Reuters news and German Frankfurter Rundschau text, four types (PER for persons, ORG, LOC and MISC for “names of miscellaneous entities that do not belong to the previous three groups”), and IOB tags: each word is B- or I- a type, or O for outside. Scoring is F1 over entities, and an entity counts only if it exactly matches the reference.
An Arabic example in IOB form:
| Token | Tag |
|---|---|
| زار | O |
| خالد | B-PER |
| العتيبي | I-PER |
| مدينة | O |
| الرياض | B-LOC |
(“Khalid Al-Otaibi visited the city of Riyadh.”)
Arabic has its own datasets. ANERcorp, created by Yassine Benajiba and a standard reference since 2008, is newswire text; CAMeL Lab’s agreed split has 125,102 training words and 25,008 test words under CC BY-SA 4.0. Wojood (Jarrar et al., 2022) has about 550K tokens of MSA and dialect text, 21 entity types and about 75K entities, 22.5% of them nested. Its AraBERT-based model scored 0.884 micro F1.
With an LLM, the common framing is different: ask for a list, not a tag per token.
{"entities": [
{"text": "خالد العتيبي", "type": "PERSON"},
{"text": "الرياض", "type": "LOCATION"}
]}
Why it matters
NER feeds search filters, CRM records, redaction of PII, and knowledge graphs. With an LLM, the main risks are:
- Invented or edited entities. A model can return a name that is not in the text, or “correct” its spelling. Check every entity appears in the source, after Arabic text normalization on both sides. This is a form of hallucination check.
- Label drift. Fix the type list in a schema, so the model cannot invent a new type.
- Benchmarks are not your data. Public labels rarely match yours. Hand-label a sample and score precision, recall and F1, as in any eval.
On BroadNet Inference
BroadNet has no NER endpoint; you build it on POST /v1/chat/completions. BroadNet’s API reference documents structured output for GLM 5.3 and Qwen3.8 27B, which is how you hold the model to an entity list and a fixed set of types. For documents that arrive as images, Qwen3.8 27B also takes image input.
BroadNet publishes no NER benchmark for its models. The Arabic named entity extraction post has a prompt, span checks that ignore diacritics, Hijri date handling and a scoring script to run on your own sample.