LlamaIndex is a Python framework for RAG and agents. For any OpenAI-compatible API that isn’t OpenAI itself, it provides the OpenAILike class. BroadNet Inference serves standard chat completions at https://inference.broadnet.ai/v1. BroadRouter is BroadNet’s own site; this follows the LlamaIndex API reference for llama-index-llms-openai-like 0.9.0.
Install
pip install llama-index-llms-openai-like
export BROADNET_API_KEY='your-broadnet-key'
Connect and test
import os
from llama_index.llms.openai_like import OpenAILike
llm = OpenAILike(
model="broadnet/glm-5.3",
api_base="https://inference.broadnet.ai/v1",
api_key=os.environ["BROADNET_API_KEY"],
context_window=1048576,
is_chat_model=True,
is_function_calling_model=True,
max_tokens=2000, # reasoning counts toward this limit
)
print(llm.complete("Name three uses for a vector index."))
A printed answer means the key, URL and model ID work.
The four settings that matter
| Parameter | Set it to | Default if you don’t |
|---|---|---|
is_chat_model |
True |
False, which uses the legacy completions endpoint; BroadNet serves chat completions |
context_window |
The model’s real context | 3,900 tokens |
is_function_calling_model |
True only for GLM 5.3 or Qwen3.8 27B |
False |
max_tokens |
1,000 or more | None (not sent, so the server decides) |
is_chat_model matters most. The API reference says it “controls whether the model uses the chat or completion endpoint” and defaults to false. BroadNet’s docs list /v1/chat/completions, not the older completions endpoint.
Use it across an app
from llama_index.core import Settings
Settings.llm = llm
Query engines, chat engines and agents then use BroadNet. Embeddings are separate: BroadNet’s API reference doesn’t list an embeddings endpoint, so keep a different embedding model for indexing.
Context windows
| Model | ID | context_window |
Tools |
|---|---|---|---|
| GLM 5.3 | broadnet/glm-5.3 |
1048576 | Yes |
| DeepSeek V4.1 Flash | broadnet/deepseek-v4.1-flash |
1048576 | Not listed |
| Qwen3.8 27B | broadnet/qwen3.8-27b |
262144 | Yes |
| BroadNet Gemma 4 VL 26B | broadnet/gemma-4-26b-a4b |
32768 | Not listed |
A 1M-token context lets a query engine pass many more retrieved chunks per call. That isn’t always better; see long context vs RAG.
Troubleshooting
- 401. The key is wrong, expired or out of allowance.
- 404.
api_basemust end in/v1, andis_chat_modelmust beTrue. Then check the model ID. Paid models need a top-up; the $5 credit covers onlybroadnet/gemma-4-26b-a4band the two Transcribe models. - Empty answers. Reasoning used the output budget. Raise
max_tokens. Why answers come back empty. - The agent never calls tools. Set
is_function_calling_model=Trueand usebroadnet/glm-5.3orbroadnet/qwen3.8-27b. - 429 during ingestion or batch queries. A pay-as-you-go key allows 30 requests a minute and 4 in flight. OpenAILike retries 3 times by default (
max_retries). See errors and retries.
