---
title: "What is BM25? Okapi BM25 keyword ranking explained | BroadRouter"
description: "BM25 is the standard keyword ranking function in Lucene and Elasticsearch. How its k1 and b settings work, and why Arabic text needs normalization first."
url: https://broadrouter.com/glossary/bm25/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# BM25

Updated 2026-11-16

BM25 (Okapi BM25) is a ranking function for keyword search that scores a document by how often the query terms appear in it, how rare those terms are across the collection, and how long the document is. It is the default similarity in Lucene and Elasticsearch.

Also called: Okapi BM25, lexical search

BM25 is the keyword half of most search systems: fast, cheap and good at exact terms.

## How it works

BM25 comes from the probabilistic relevance framework, summarized by Robertson and Zaragoza in “The Probabilistic Relevance Framework: BM25 and Beyond” (2009). Lucene’s class notes that it was introduced by Robertson, Walker, Jones and others at TREC-3.

For each query term, BM25 combines three signals:

-   **Term frequency**: more occurrences in the document raise the score, but with diminishing returns.
-   **Inverse document frequency (IDF)**: rare terms count more. Lucene implements it as `log(1 + (docCount - docFreq + 0.5)/(docFreq + 0.5))`.
-   **Length normalization**: a match in a short document counts more than the same match in a long one.

Two parameters tune it. Elasticsearch’s docs:

| Parameter | Default | What it controls |
| --- | --- | --- |
| `k1` | 1.2 | “non-linear term frequency normalization (saturation)” |
| `b` | 0.75 | “to what degree document length normalizes tf values” |

BM25 is the default similarity in Elasticsearch.

## Why it matters

-   **Exact matches.** Product codes, names and error messages are where [embeddings](https://broadrouter.com/glossary/embeddings/) are weakest and BM25 is strong.
-   **Cost.** It needs no model calls or GPU, only an inverted index.
-   **Hybrid search.** Most production RAG systems pair it with vector search; see [hybrid search](https://broadrouter.com/glossary/hybrid-search/).

## Arabic text and BM25

BM25 matches tokens exactly, so it depends on how text is analyzed. In Arabic, the same word can be written with different alef forms, with or without diacritics, or with teh marbuta (ة) written as heh (ه), and articles and conjunctions attach to the word. Without [Arabic text normalization](https://broadrouter.com/glossary/arabic-text-normalization/) and stemming, those variants do not match.

Elasticsearch’s built-in `arabic` analyzer chains these filters: `lowercase`, `decimal_digit`, `arabic_stop`, `arabic_normalization`, a keyword marker, and `arabic_stemmer`. Normalize queries and documents the same way. [Arabic text normalization for search](https://broadrouter.com/blog/arabic-text-normalization-search/) tests which rules help and which hurt, and the [Arabic text normalizer](https://broadrouter.com/tools/arabic-text-normalizer/) tool lets you try them on your own text.

## On BroadNet Inference

BM25 runs in your search engine, not in the LLM. BroadNet does not offer embeddings or rerank endpoints; it serves the chat models that read what you retrieve. Send your top BM25 or hybrid results to a model such as [Qwen3.8 27B](https://broadrouter.com/models/qwen3.8-27b/) over the Chat Completions API. [Arabic RAG search](https://broadrouter.com/blog/arabic-rag-search/) shows a full pipeline.

## Questions

### Is BM25 still useful when I have embeddings?

Yes. BM25 is strong on exact terms such as names, codes and error strings that embeddings can blur. Many RAG systems run both and merge the results with hybrid search.

### What do k1 and b do?

k1 controls how quickly repeated terms stop adding score (term frequency saturation). b controls how much document length is normalized. Lucene and Elasticsearch default to k1 1.2 and b 0.75.

### Does BM25 work for Arabic?

Yes, if the text is analyzed for Arabic first. Without normalization and stemming, spelling variants and attached prefixes stop words from matching.

## See also

-   [Hybrid search](https://broadrouter.com/glossary/hybrid-search/) (term)
-   [Embeddings](https://broadrouter.com/glossary/embeddings/) (term)
-   [Reranking](https://broadrouter.com/glossary/reranking/) (term)
-   [RAG (retrieval-augmented generation)](https://broadrouter.com/glossary/rag/) (term)
-   [Arabic text normalization](https://broadrouter.com/glossary/arabic-text-normalization/) (term)
-   [Vector database](https://broadrouter.com/glossary/vector-database/) (term)
-   [Arabic text normalization for search and RAG: what to change, what not](https://broadrouter.com/blog/arabic-text-normalization-search/) (article)
-   [Arabic RAG: normalisation, chunking and retrieval that works](https://broadrouter.com/blog/arabic-rag-search/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-bm25&utm_content=page-cta) ·

**Sources**

-   [Elasticsearch docs, similarity module (BM25 default, k1 and b) (checked 2026-11-15)](https://www.elastic.co/docs/reference/elasticsearch/index-settings/similarity)
-   [Apache Lucene 9.12 API, BM25Similarity (checked 2026-11-15)](https://lucene.apache.org/core/9_12_0/core/org/apache/lucene/search/similarities/BM25Similarity.html)
-   [Elasticsearch docs, language analyzers (arabic analyzer) (checked 2026-11-15)](https://www.elastic.co/docs/reference/text-analysis/analysis-lang-analyzer)
-   [Robertson and Zaragoza, The Probabilistic Relevance Framework: BM25 and Beyond, Foundations and Trends in Information Retrieval 3, 2009 (DOI record, checked 2026-11-15)](https://doi.org/10.1561/1500000019)
