LangChain’s ChatOpenAI accepts any endpoint through base_url. This works with BroadNet Inference because it uses the standard chat-completions shape. If you call the API directly instead, see the Python SDK guide.
Install
Current langchain-openai releases (1.x) need Python 3.11 or newer.
pip install -U langchain-openai
export BROADNET_API_KEY='paste-the-key-once'
Connect
import os
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
model="broadnet/glm-5.3",
base_url="https://inference.broadnet.ai/v1",
api_key=os.environ["BROADNET_API_KEY"],
max_tokens=2000, # reasoning counts toward this limit
)
print(llm.invoke("Summarise the CAP theorem in two sentences.").content)
openai_api_base is an alias for base_url. Without either, LangChain reads OPENAI_API_BASE, and then the underlying openai client reads OPENAI_BASE_URL.
Things to know
- Reasoning text isn’t kept. LangChain’s docs say
ChatOpenAIfollows the official OpenAI specification and does not keep non-standard fields such asreasoning_content. You get the final answer, which is usually what a chain needs. - Leave room for reasoning. GLM 5.3, DeepSeek V4.1 Flash and Qwen3.8 27B reason first. A small
max_tokenscan return an empty answer. Why answers come back empty. - Streaming usage. When you configure the endpoint with
OPENAI_BASE_URL, LangChain turns offstream_usageby default. Passstream_usage=Trueif you want token counts while streaming. BroadNet’s docs say usage arrives in the last chunk when it’s requested. - Stay on chat completions. BroadNet serves
/v1/chat/completions, not the Responses API. LangChain’s docs sayChatOpenAIswitches to the Responses API when you use OpenAI built-in tools (web search, file search, code interpreter and similar), conversation state, or thereasoningparameter. Leave those out, and don’t setuse_responses_api=True. - Token limit name. LangChain’s docs say
max_tokensis converted tomax_completion_tokensinternally. BroadNet’s docs describemax_tokens. If a limit seems ignored, check the request your app sends.
Which model for which chain
| Chain | Model |
|---|---|
| Tool calling and structured output | broadnet/glm-5.3 |
| Long documents and agents, unlimited plan | broadnet/deepseek-v4.1-flash |
| Arabic text or images | broadnet/qwen3.8-27b |
Troubleshooting
- 401. The key is wrong, expired, or out of pay-as-you-go allowance. If you passed no
api_key, LangChain usedOPENAI_API_KEY, which may hold an OpenAI key. - 404 or “not found”. Check that
base_urlends in/v1and that nothing routes the call to the Responses API (see above). Then check the model ID matches BroadNet’s exactly, with thebroadnet/prefix. Paid models need a top-up; the $5 credit coversbroadnet/gemma-4-26b-a4band the two Transcribe models. - Empty
content. Reasoning used the budget. Raisemax_tokens. Why answers come back empty. - Tool calls never happen. Use
broadnet/glm-5.3orbroadnet/qwen3.8-27b. BroadNet’s docs list tools only for those two. - 429 in batch or parallel chains. A pay-as-you-go key allows 30 requests a minute and 4 at once. Set
max_concurrencyin the runnable config forbatch(). See error codes and retries.
