---
title: "LangGraph with a custom OpenAI-compatible base_url | BroadRouter"
description: "Run a LangGraph StateGraph or create_agent on BroadNet Inference with ChatOpenAI, base_url and a BroadNet model ID, plus tools, streaming and memory."
url: https://broadrouter.com/guides/langgraph-openai-compatible/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Build LangGraph agents on BroadNet Inference

Updated 2026-11-16

Install langgraph and langchain-openai, then create ChatOpenAI with model set to a BroadNet ID such as broadnet/glm-5.3, base\_url set to https://inference.broadnet.ai/v1 and api\_key set to your BroadNet key. Bind your tools to it and use it in a StateGraph node, or pass it to langchain's create\_agent. Use GLM 5.3 or Qwen3.8 27B, the models BroadNet lists with tools.

LangGraph is LangChain’s library for stateful, multi-step agents built as graphs. It has no model client of its own: your nodes call a LangChain chat model. To run a graph on BroadNet Inference, give it `ChatOpenAI` from `langchain-openai` with BroadNet’s base URL. BroadRouter is BroadNet’s own site; this follows the LangGraph and LangChain docs as of LangGraph 1.2.22 and langchain-openai 1.8.

## Install

```bash
pip install langgraph langchain-openai
export BROADNET_API_KEY='your-broadnet-key'
```

## A tool-calling graph

```python
import os
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langgraph.graph import StateGraph, MessagesState, START
from langgraph.prebuilt import ToolNode, tools_condition
from langgraph.checkpoint.memory import InMemorySaver

@tool
def get_weather(city: str) -> str:
    """Get the weather for a city."""
    return f"21C and sunny in {city}"

tools = [get_weather]
model = ChatOpenAI(
    model="broadnet/glm-5.3",
    base_url="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
).bind_tools(tools)

def call_model(state: MessagesState):
    return {"messages": [model.invoke(state["messages"])]}

builder = StateGraph(MessagesState)
builder.add_node("model", call_model)
builder.add_node("tools", ToolNode(tools))
builder.add_edge(START, "model")
builder.add_conditional_edges("model", tools_condition)
builder.add_edge("tools", "model")
graph = builder.compile(checkpointer=InMemorySaver())

config = {"configurable": {"thread_id": "demo-1"}}
result = graph.invoke({"messages": [{"role": "user", "content": "What's the weather in Riyadh?"}]}, config)
print(result["messages"][-1].content)
```

The model node calls BroadNet. `tools_condition` sends the graph to the tool node when the reply has tool calls, and the tool result goes back to the model. The checkpointer keeps the conversation per `thread_id`, so a second `invoke` with the same config continues it.

## Stream tokens from the graph

```python
for chunk, meta in graph.stream(
    {"messages": [{"role": "user", "content": "And in Dubai?"}]},
    config,
    stream_mode="messages",
):
    if chunk.content and meta["langgraph_node"] == "model":
        print(chunk.content, end="", flush=True)
```

With `stream_mode="messages"`, `ChatOpenAI` sends `stream: true` to BroadNet and LangGraph passes the tokens through.

## The prebuilt agent

If you don’t need a custom graph, the prebuilt agent loop takes the same model:

```python
from langchain.agents import create_agent  # pip install langchain

agent = create_agent(
    ChatOpenAI(model="broadnet/glm-5.3", base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"]),
    tools=tools,
)
agent.invoke({"messages": [{"role": "user", "content": "Weather in Riyadh?"}]})
```

`create_react_agent` from `langgraph.prebuilt` still runs in LangGraph 1.2.22, but prints a warning that it has moved to `langchain.agents.create_agent`.

## Which model to pick

| Graph | Model |
| --- | --- |
| Agents with tools, structured output | `broadnet/glm-5.3` (1,048,576-token context) |
| Tools plus image input or Arabic text | `broadnet/qwen3.8-27b` (262,144-token context) |
| Graphs with no tool calls, long documents | `broadnet/deepseek-v4.1-flash` |

BroadNet’s docs list tools and structured output for GLM 5.3 and Qwen3.8 27B only. DeepSeek V4.1 Flash is the model on the [unlimited plans](https://broadrouter.com/guides/unlimited-llm-api-sessions/), so it suits summarise-and-route graphs that don’t call tools. Test a tool loop on it before you rely on it. [Prices for every model](https://broadrouter.com/pricing/).

## Settings that matter

-   **Stay on chat completions.** BroadNet serves `/v1/chat/completions`. LangChain’s docs say `ChatOpenAI` moves to the Responses API when you use OpenAI built-in tools, conversation state or the `reasoning` parameter. Leave those out and don’t set `use_responses_api=True`.
-   **Output limit.** Reasoning counts toward the output budget on DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B. Set `max_tokens` to 2,000 or more on the model. In our test, langchain-openai 1.8 sent it as `max_completion_tokens`.
-   **Parallel branches.** Each branch that calls the model is a separate request. A pay-as-you-go key allows 4 requests in flight, and a plan session runs one at a time.

## How we tested this

On 2026-11-15 we installed langgraph 1.2.22, langchain-openai 1.8.0 and langchain 1.5.0 in a clean virtual environment. We ran the graph above, the streaming loop and `create_agent` against a local mock server that speaks BroadNet’s chat-completions shape and returns tool calls. We checked the logged requests. Every call went to `/v1/chat/completions` with the model ID `broadnet/glm-5.3` unchanged, a Bearer key and the tool schema, and the tool result came back as a `tool` message. We had no BroadNet key, so we did not test real model output.

## Troubleshooting

-   **401.** The key is wrong, expired or out of allowance.
-   **404.** `base_url` must end in `/v1`. Check the model ID, with the `broadnet/` prefix. Paid models need a top-up; the $5 credit covers only `broadnet/gemma-4-26b-a4b` and the two Transcribe models.
-   **The agent never calls a tool.** Switch to `broadnet/glm-5.3` or `broadnet/qwen3.8-27b`.
-   **Empty answers.** The output budget went on reasoning. Raise `max_tokens`. [Why answers come back empty](https://broadrouter.com/guides/reasoning-tokens-empty-response/).
-   **429.** A pay-as-you-go key allows 30 requests a minute. Add `max_retries` to `ChatOpenAI` and see [errors and retries](https://broadrouter.com/guides/errors-and-retries/).

## Terms in this guide

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

[Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/)

Pay-as-you-go is a pricing model in which you pay only for what you use, metered per unit such as tokens, requests or audio minutes, with no subscription or minimum commitment. Most LLM APIs bill this way, from a prepaid balance or a periodic invoice.

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Structured output](https://broadrouter.com/glossary/structured-output/)

Structured output is an LLM API feature that constrains a model's reply to a format the developer specifies, usually a JSON Schema, so code can parse it reliably. It is stricter than JSON mode, which guarantees valid JSON but not a particular shape.

[AI agent](https://broadrouter.com/glossary/agent/)

An AI agent is a system in which a language model directs its own steps toward a goal: it decides which tools to call, reads the results and repeats until it produces a final answer or hits a limit. A fixed script that calls a model at set points is usually called a workflow instead.

[Base URL](https://broadrouter.com/glossary/base-url/)

A base URL is the fixed root address of an API, such as https://api.example.com/v1, that a client puts in front of every endpoint path, such as /chat/completions. In OpenAI-compatible SDKs, changing the base URL, together with the API key and model ID, points the same code at a different provider.

[Chat Completions API](https://broadrouter.com/glossary/chat-completions-api/)

The Chat Completions API is OpenAI's HTTP endpoint, POST /v1/chat/completions, that takes a model name and a list of role-tagged messages and returns the model's next message. Its request and response shape became the common format that OpenAI-compatible providers and SDKs reuse.

[Concurrency](https://broadrouter.com/glossary/concurrency/)

Concurrency is the number of requests a client has in flight at the same time. LLM APIs often cap it per key or plan, separately from requests per minute, because each open request holds GPU capacity until its response has finished.

## Questions

### Does LangGraph need its own BroadNet integration?

No. LangGraph runs whatever chat model your nodes call. Use LangChain's ChatOpenAI with base\_url set to https://inference.broadnet.ai/v1, and every node that calls the model goes to BroadNet over chat completions.

### Should I use create\_react\_agent or create\_agent?

In LangGraph 1.2, create\_react\_agent from langgraph.prebuilt still works but prints a deprecation warning that points to create\_agent in the langchain package. Both worked against our mock server with a BroadNet-style model ID.

### Which BroadNet model should a LangGraph agent use?

GLM 5.3 (broadnet/glm-5.3) or Qwen3.8 27B (broadnet/qwen3.8-27b). An agent loop depends on tool calls, and BroadNet lists tools for these two models only.

### Why is the max\_tokens I set sent as max\_completion\_tokens?

langchain-openai 1.8 renames max\_tokens to max\_completion\_tokens in the request. BroadNet's API reference describes max\_tokens. If an output limit seems to be ignored, check the request your app actually sends.

## Related guides

-   [Use LangChain with BroadNet Inference](https://broadrouter.com/guides/langchain-openai-base-url/)
-   [Run the OpenAI Agents SDK on BroadNet Inference](https://broadrouter.com/guides/openai-agents-sdk-custom-provider/)
-   [Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)
-   [BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/)
-   [All tools that work with BroadNet Inference](https://broadrouter.com/integrations/): SDKs, coding agents, chat UIs and automation tools, with exact settings for each

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-langgraph-openai-compatible&utm_content=page-cta) ·

**Sources**

-   [LangGraph docs, Quickstart and Graph API (checked 2026-11-15)](https://docs.langchain.com/oss/python/langgraph/graph-api)
-   [LangChain docs, ChatOpenAI integration (checked 2026-11-15)](https://docs.langchain.com/oss/python/integrations/chat/openai)
-   [PyPI, langgraph 1.2.22 (checked 2026-11-15)](https://pypi.org/project/langgraph/)
-   [BroadNet Inference API reference (checked 2026-11-15)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-langgraph-openai-compatible&utm_content=sources)
