BroadRouter
Get started

Use DSPy with BroadNet Inference (set model_type="chat")

Updated

Create dspy.LM with the model "openai/broadnet/glm-5.3", model_type="chat", api_base set to https://inference.broadnet.ai/v1 and your BroadNet api_key, then pass it to dspy.configure. The model_type="chat" part matters. DSPy 3.5 defaults OpenAI-prefixed models to the Responses API, which BroadNet doesn't serve. In our mock-server test, leaving it out sent the call to /v1/responses, which got a 404, and BroadNet's live endpoint also answers that path with a 404.

DSPy is a Python framework for programming language models with signatures and modules, and for optimizing the prompts behind them. You connect it to a model with dspy.LM. DSPy’s docs show OpenAI-compatible servers as an openai/ model with api_base. For BroadNet Inference, which serves chat completions at https://inference.broadnet.ai/v1, you also need model_type="chat". BroadRouter is BroadNet’s own site; this follows the DSPy docs for 3.5.

Install

pip install dspy
export BROADNET_API_KEY='your-broadnet-key'

Connect

import os
import dspy

lm = dspy.LM(
    "openai/broadnet/glm-5.3",
    model_type="chat",
    api_base="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
    max_tokens=4000,
)
dspy.configure(lm=lm)

qa = dspy.Predict("question -> answer")
print(qa(question="What is the capital of Saudi Arabia?").answer)
  • openai/ tells DSPy to use its OpenAI client. The rest of the string, broadnet/glm-5.3, is the model ID sent to BroadNet.
  • model_type="chat" keeps DSPy on /v1/chat/completions. DSPy’s LM reference says the default, 'auto', makes OpenAI models “use Responses”.
  • max_tokens leaves room for reasoning. DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B think before they answer, and the thinking counts toward the limit.

What happens without model_type

The example on DSPy’s installation page for an OpenAI-compatible server leaves model_type out. In our test with DSPy 3.5.0, that sent the request to /v1/responses with max_output_tokens, and the call failed:

dspy.utils.exceptions.LMInvalidRequestError: [openai/broadnet/glm-5.3] /v1/responses not served

That error came from our mock server. BroadNet’s live endpoint also answered /v1/responses with a 404 when we checked on 2026-11-16. Adding model_type="chat" fixed it.

Modules and models

cot = dspy.ChainOfThought("question -> answer")
print(cot(question="A train leaves at 9:40 and arrives at 13:05. How long is the trip?").answer)
DSPy use Model
Predict, ChainOfThought, optimizer runs broadnet/deepseek-v4.1-flash (low per-token price; flat-price plans)
ReAct and other tool-calling modules broadnet/glm-5.3 or broadnet/qwen3.8-27b
Trying DSPy on the starting credit broadnet/gemma-4-26b-a4b (32,768-token context)

BroadNet’s docs list tools and structured output for GLM 5.3 and Qwen3.8 27B only. You can use one model for the program and another for an optimizer, each with its own dspy.LM. Prices for every model.

How we tested this

On 2026-11-15 we installed dspy 3.5.0 in a clean virtual environment and ran dspy.Predict against a local mock server that speaks BroadNet’s chat-completions shape and answers /v1/responses with a 404. Without model_type, DSPy called /v1/responses and failed. With model_type="chat", it called /v1/chat/completions with the model ID broadnet/glm-5.3 and a Bearer key, and parsed the answer field. We saw that DSPy sent the output limit as max_completion_tokens. BroadNet’s API reference describes max_tokens and doesn’t mention max_completion_tokens, so we can’t confirm that BroadNet applies a limit sent that way. We tested against a mock server, not with a BroadNet key, so we did not check real model output.

Troubleshooting

  • 404 on /v1/responses. Add model_type="chat".
  • 404 on /v1/chat/completions. api_base must end in /v1, and the model string must be openai/ plus the full BroadNet ID. Paid models need a top-up; the $5 credit covers only broadnet/gemma-4-26b-a4b and the two Transcribe models.
  • “Expected to find output fields”. The answer was empty or cut short, often because reasoning used the output budget. Raise max_tokens. Why answers come back empty.
  • 429 during optimization. A pay-as-you-go key allows 30 requests a minute and 4 in flight. Lower the optimizer’s thread count and see errors and retries.

Questions

Why does DSPy call /v1/responses on BroadNet?

DSPy 3.5's dspy.LM reference says model_type 'auto', the default, makes OpenAI models use the Responses API. An openai/ model with a custom api_base is treated the same way. Pass model_type="chat" to use chat completions, which is what BroadNet serves.

How do I write the BroadNet model ID in DSPy?

Put openai/ in front of the BroadNet ID, for example openai/broadnet/glm-5.3. DSPy uses openai/ to pick the client and, in our test, sent broadnet/glm-5.3 as the model.

Which BroadNet model suits DSPy optimizers?

Optimizers make many calls, so cost and rate limits matter. DeepSeek V4.1 Flash has a low per-token price and is the model on BroadNet's flat-price plans, which aren't metered per token. Use GLM 5.3 or Qwen3.8 27B for modules that call tools, such as ReAct.

Does DSPy's cache matter with BroadNet?

DSPy caches LM calls by default (cache=True), so repeating the same call costs nothing. Set cache=False while you are checking the connection, or you may be looking at a cached answer.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity