---
title: "LlamaIndex OpenAILike with a custom api_base | BroadRouter"
description: "Connect LlamaIndex to BroadNet Inference with the OpenAILike class, setting api_base, context_window, is_chat_model and is_function_calling_model correctly."
url: https://broadrouter.com/guides/llamaindex-openailike/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Use LlamaIndex with BroadNet Inference through OpenAILike

Updated 2026-11-16

Install llama-index-llms-openai-like and create OpenAILike with model set to a BroadNet ID such as broadnet/glm-5.3, api\_base set to https://inference.broadnet.ai/v1, your BroadNet key, context\_window set to the model's real context, is\_chat\_model=True, and is\_function\_calling\_model=True only for GLM 5.3 or Qwen3.8 27B. Set it as Settings.llm to use it everywhere.

LlamaIndex is a Python framework for RAG and agents. For any OpenAI-compatible API that isn’t OpenAI itself, it provides the `OpenAILike` class. BroadNet Inference serves standard chat completions at `https://inference.broadnet.ai/v1`. BroadRouter is BroadNet’s own site; this follows the LlamaIndex API reference for `llama-index-llms-openai-like` 0.9.0.

## Install

```bash
pip install llama-index-llms-openai-like
export BROADNET_API_KEY='your-broadnet-key'
```

## Connect and test

```python
import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="broadnet/glm-5.3",
    api_base="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
    context_window=1048576,
    is_chat_model=True,
    is_function_calling_model=True,
    max_tokens=2000,  # reasoning counts toward this limit
)

print(llm.complete("Name three uses for a vector index."))
```

A printed answer means the key, URL and model ID work.

## The four settings that matter

| Parameter | Set it to | Default if you don’t |
| --- | --- | --- |
| `is_chat_model` | `True` | `False`, which uses the legacy completions endpoint; BroadNet serves chat completions |
| `context_window` | The model’s real context | 3,900 tokens |
| `is_function_calling_model` | `True` only for GLM 5.3 or Qwen3.8 27B | `False` |
| `max_tokens` | 1,000 or more | None (not sent, so the server decides) |

`is_chat_model` matters most. The API reference says it “controls whether the model uses the chat or completion endpoint” and defaults to false. BroadNet’s docs list `/v1/chat/completions`, not the older completions endpoint.

## Use it across an app

```python
from llama_index.core import Settings

Settings.llm = llm
```

Query engines, chat engines and agents then use BroadNet. Embeddings are separate: BroadNet’s API reference doesn’t list an embeddings endpoint, so keep a different embedding model for indexing.

## Context windows

| Model | ID | `context_window` | Tools |
| --- | --- | --- | --- |
| GLM 5.3 | `broadnet/glm-5.3` | 1048576 | Yes |
| DeepSeek V4.1 Flash | `broadnet/deepseek-v4.1-flash` | 1048576 | Not listed |
| Qwen3.8 27B | `broadnet/qwen3.8-27b` | 262144 | Yes |
| BroadNet Gemma 4 VL 26B | `broadnet/gemma-4-26b-a4b` | 32768 | Not listed |

A 1M-token context lets a query engine pass many more retrieved chunks per call. That isn’t always better; see [long context vs RAG](https://broadrouter.com/blog/long-context-vs-rag/).

## Troubleshooting

-   **401.** The key is wrong, expired or out of allowance.
-   **404.** `api_base` must end in `/v1`, and `is_chat_model` must be `True`. Then check the model ID. Paid models need a top-up; the $5 credit covers only `broadnet/gemma-4-26b-a4b` and the two Transcribe models.
-   **Empty answers.** Reasoning used the output budget. Raise `max_tokens`. [Why answers come back empty](https://broadrouter.com/guides/reasoning-tokens-empty-response/).
-   **The agent never calls tools.** Set `is_function_calling_model=True` and use `broadnet/glm-5.3` or `broadnet/qwen3.8-27b`.
-   **429 during ingestion or batch queries.** A pay-as-you-go key allows 30 requests a minute and 4 in flight. OpenAILike retries 3 times by default (`max_retries`). See [errors and retries](https://broadrouter.com/guides/errors-and-retries/).

## Terms in this guide

[Embeddings](https://broadrouter.com/glossary/embeddings/)

Embeddings are lists of floating-point numbers (vectors) that a model produces to represent a piece of text, an image or other input. Inputs with similar meaning get vectors that are close together, so distance between vectors can rank search results, cluster documents or find related items.

[Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/)

Pay-as-you-go is a pricing model in which you pay only for what you use, metered per unit such as tokens, requests or audio minutes, with no subscription or minimum commitment. Most LLM APIs bill this way, from a prepaid balance or a periodic invoice.

[Chat Completions API](https://broadrouter.com/glossary/chat-completions-api/)

The Chat Completions API is OpenAI's HTTP endpoint, POST /v1/chat/completions, that takes a model name and a list of role-tagged messages and returns the model's next message. Its request and response shape became the common format that OpenAI-compatible providers and SDKs reuse.

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

[Concurrency](https://broadrouter.com/glossary/concurrency/)

Concurrency is the number of requests a client has in flight at the same time. LLM APIs often cap it per key or plan, separately from requests per minute, because each open request holds GPU capacity until its response has finished.

[Context window](https://broadrouter.com/glossary/context-window/)

A context window is the maximum number of tokens a language model can work with in a single request. It covers the prompt, conversation history, tool definitions, images and the generated output, including any reasoning tokens. Anything that does not fit must be cut, summarised or split across several requests.

[OpenAI-compatible API](https://broadrouter.com/glossary/openai-compatible-api/)

An OpenAI-compatible API is an HTTP API that accepts the same request and response formats as the OpenAI API, such as POST /v1/chat/completions, so OpenAI SDKs and tools can use it by changing only the base URL, the API key and the model name.

## Questions

### Why not use LlamaIndex's OpenAI class with a different api\_base?

LlamaIndex's own OpenAI page says to use the OpenAILike integration for any OpenAI-compatible API that is not the official OpenAI API. OpenAILike also lets you declare the context window and tool support yourself.

### What happens if I leave context\_window out?

OpenAILike defaults to 3,900 tokens, so LlamaIndex trims prompts and retrieved chunks far below what BroadNet's models accept. Set 1048576 for GLM 5.3 and DeepSeek V4.1 Flash, or 262144 for Qwen3.8 27B.

### Can a LlamaIndex FunctionAgent use BroadNet?

Yes, with is\_function\_calling\_model=True on a model BroadNet lists with tools, GLM 5.3 or Qwen3.8 27B. Leave it False for DeepSeek V4.1 Flash, where BroadNet's docs don't list tool calling.

### Does BroadNet provide embeddings for a LlamaIndex index?

BroadNet's API reference lists chat completions and audio transcriptions, not embeddings. Use a separate embedding model for the index and BroadNet for the LLM.

## Related guides

-   [Use LangChain with BroadNet Inference](https://broadrouter.com/guides/langchain-openai-base-url/)
-   [Use Pydantic AI with BroadNet Inference (OpenAICompatibleProvider)](https://broadrouter.com/guides/pydantic-ai-openai-compatible/)
-   [Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)
-   [Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)
-   [All tools that work with BroadNet Inference](https://broadrouter.com/integrations/): SDKs, coding agents, chat UIs and automation tools, with exact settings for each

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-llamaindex-openailike&utm_content=page-cta) ·

**Sources**

-   [LlamaIndex API reference, OpenAILike (checked 2026-11-15)](https://developers.llamaindex.ai/python/framework-api-reference/llms/openai_like/)
-   [LlamaIndex docs, OpenAILike example (Grok integration page) (checked 2026-11-15)](https://developers.llamaindex.ai/python/framework/integrations/llm/grok/)
-   [LlamaIndex docs, OpenAI LLM page (points compatible APIs to OpenAILike) (checked 2026-11-15)](https://developers.llamaindex.ai/python/framework/integrations/llm/openai/)
-   [PyPI, llama-index-llms-openai-like 0.9.0, Python 3.11+ (checked 2026-11-15)](https://pypi.org/project/llama-index-llms-openai-like/)
-   [LlamaIndex source, OpenAI LLM max\_tokens default and payload (checked 2026-11-16)](https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/base.py)
-   [BroadNet Inference API reference (checked 2026-11-15)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-llamaindex-openailike&utm_content=sources)
