BroadRouter
Get started

Named entity recognition (NER)

Updated

Named entity recognition (NER) is the task of finding names in text and labelling each with a type, such as person, organization or location. Classic systems tag every token; LLMs are often asked instead to return a list of entities as JSON, which is then checked against the source.

Also called: NER, named entity recognition, entity extraction, named entities

Pull every person, company and place out of a contract or a news story, with a type for each. That is NER, and it is one of the most common jobs given to an LLM.

How it works

The classic benchmark is the CoNLL-2003 shared task. It used English Reuters news and German Frankfurter Rundschau text, four types (PER for persons, ORG, LOC and MISC for “names of miscellaneous entities that do not belong to the previous three groups”), and IOB tags: each word is B- or I- a type, or O for outside. Scoring is F1 over entities, and an entity counts only if it exactly matches the reference.

An Arabic example in IOB form:

Token Tag
زار O
خالد B-PER
العتيبي I-PER
مدينة O
الرياض B-LOC

(“Khalid Al-Otaibi visited the city of Riyadh.”)

Arabic has its own datasets. ANERcorp, created by Yassine Benajiba and a standard reference since 2008, is newswire text; CAMeL Lab’s agreed split has 125,102 training words and 25,008 test words under CC BY-SA 4.0. Wojood (Jarrar et al., 2022) has about 550K tokens of MSA and dialect text, 21 entity types and about 75K entities, 22.5% of them nested. Its AraBERT-based model scored 0.884 micro F1.

With an LLM, the common framing is different: ask for a list, not a tag per token.

{"entities": [
  {"text": "خالد العتيبي", "type": "PERSON"},
  {"text": "الرياض", "type": "LOCATION"}
]}

Why it matters

NER feeds search filters, CRM records, redaction of PII, and knowledge graphs. With an LLM, the main risks are:

  • Invented or edited entities. A model can return a name that is not in the text, or “correct” its spelling. Check every entity appears in the source, after Arabic text normalization on both sides. This is a form of hallucination check.
  • Label drift. Fix the type list in a schema, so the model cannot invent a new type.
  • Benchmarks are not your data. Public labels rarely match yours. Hand-label a sample and score precision, recall and F1, as in any eval.

On BroadNet Inference

BroadNet has no NER endpoint; you build it on POST /v1/chat/completions. BroadNet’s API reference documents structured output for GLM 5.3 and Qwen3.8 27B, which is how you hold the model to an entity list and a fixed set of types. For documents that arrive as images, Qwen3.8 27B also takes image input.

BroadNet publishes no NER benchmark for its models. The Arabic named entity extraction post has a prompt, span checks that ignore diacritics, Hijri date handling and a scoring script to run on your own sample.

Questions

What is the difference between NER and entity extraction?

NER in research usually means tagging every token with a label. Entity extraction often means returning a list of entities, which is how most LLM pipelines frame it. Score the framing you actually ship.

Why does an entity the model returned not match my Arabic text?

Often the model changed a hamza, removed tatweel or added diacritics while copying. Normalize both the entity and the source before checking that it appears.

What is nested NER?

Entities inside other entities, such as a city inside an organization name. Wojood annotates these; 22.5% of its entities are nested.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity