---
title: "DSPy dspy.LM with an OpenAI-compatible api_base | BroadRouter"
description: "Configure dspy.LM for BroadNet Inference with the openai/ prefix, api_base and model_type=\"chat\", because DSPy 3.5 sends OpenAI models to the Responses API."
url: https://broadrouter.com/guides/dspy-openai-compatible/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Use DSPy with BroadNet Inference (set model\_type="chat")

Updated 2026-11-16

Create dspy.LM with the model "openai/broadnet/glm-5.3", model\_type="chat", api\_base set to https://inference.broadnet.ai/v1 and your BroadNet api\_key, then pass it to dspy.configure. The model\_type="chat" part matters. DSPy 3.5 defaults OpenAI-prefixed models to the Responses API, which BroadNet doesn't serve. In our mock-server test, leaving it out sent the call to /v1/responses, which got a 404, and BroadNet's live endpoint also answers that path with a 404.

DSPy is a Python framework for programming language models with signatures and modules, and for optimizing the prompts behind them. You connect it to a model with `dspy.LM`. DSPy’s docs show OpenAI-compatible servers as an `openai/` model with `api_base`. For BroadNet Inference, which serves chat completions at `https://inference.broadnet.ai/v1`, you also need `model_type="chat"`. BroadRouter is BroadNet’s own site; this follows the DSPy docs for 3.5.

## Install

```bash
pip install dspy
export BROADNET_API_KEY='your-broadnet-key'
```

## Connect

```python
import os
import dspy

lm = dspy.LM(
    "openai/broadnet/glm-5.3",
    model_type="chat",
    api_base="https://inference.broadnet.ai/v1",
    api_key=os.environ["BROADNET_API_KEY"],
    max_tokens=4000,
)
dspy.configure(lm=lm)

qa = dspy.Predict("question -> answer")
print(qa(question="What is the capital of Saudi Arabia?").answer)
```

-   **`openai/`** tells DSPy to use its OpenAI client. The rest of the string, `broadnet/glm-5.3`, is the model ID sent to BroadNet.
-   **`model_type="chat"`** keeps DSPy on `/v1/chat/completions`. DSPy’s LM reference says the default, `'auto'`, makes OpenAI models “use Responses”.
-   **`max_tokens`** leaves room for reasoning. DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B think before they answer, and the thinking counts toward the limit.

## What happens without model\_type

The example on DSPy’s installation page for an OpenAI-compatible server leaves `model_type` out. In our test with DSPy 3.5.0, that sent the request to `/v1/responses` with `max_output_tokens`, and the call failed:

```text
dspy.utils.exceptions.LMInvalidRequestError: [openai/broadnet/glm-5.3] /v1/responses not served
```

That error came from our mock server. BroadNet’s live endpoint also answered `/v1/responses` with a 404 when we checked on 2026-11-16. Adding `model_type="chat"` fixed it.

## Modules and models

```python
cot = dspy.ChainOfThought("question -> answer")
print(cot(question="A train leaves at 9:40 and arrives at 13:05. How long is the trip?").answer)
```

| DSPy use | Model |
| --- | --- |
| `Predict`, `ChainOfThought`, optimizer runs | `broadnet/deepseek-v4.1-flash` (low per-token price; flat-price plans) |
| `ReAct` and other tool-calling modules | `broadnet/glm-5.3` or `broadnet/qwen3.8-27b` |
| Trying DSPy on the starting credit | `broadnet/gemma-4-26b-a4b` (32,768-token context) |

BroadNet’s docs list tools and structured output for GLM 5.3 and Qwen3.8 27B only. You can use one model for the program and another for an optimizer, each with its own `dspy.LM`. [Prices for every model](https://broadrouter.com/pricing/).

## How we tested this

On 2026-11-15 we installed dspy 3.5.0 in a clean virtual environment and ran `dspy.Predict` against a local mock server that speaks BroadNet’s chat-completions shape and answers `/v1/responses` with a 404. Without `model_type`, DSPy called `/v1/responses` and failed. With `model_type="chat"`, it called `/v1/chat/completions` with the model ID `broadnet/glm-5.3` and a Bearer key, and parsed the answer field. We saw that DSPy sent the output limit as `max_completion_tokens`. BroadNet’s API reference describes `max_tokens` and doesn’t mention `max_completion_tokens`, so we can’t confirm that BroadNet applies a limit sent that way. We tested against a mock server, not with a BroadNet key, so we did not check real model output.

## Troubleshooting

-   **404 on /v1/responses.** Add `model_type="chat"`.
-   **404 on /v1/chat/completions.** `api_base` must end in `/v1`, and the model string must be `openai/` plus the full BroadNet ID. Paid models need a top-up; the $5 credit covers only `broadnet/gemma-4-26b-a4b` and the two Transcribe models.
-   **“Expected to find output fields”.** The answer was empty or cut short, often because reasoning used the output budget. Raise `max_tokens`. [Why answers come back empty](https://broadrouter.com/guides/reasoning-tokens-empty-response/).
-   **429 during optimization.** A pay-as-you-go key allows 30 requests a minute and 4 in flight. Lower the optimizer’s thread count and see [errors and retries](https://broadrouter.com/guides/errors-and-retries/).

## Terms in this guide

[Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/)

Pay-as-you-go is a pricing model in which you pay only for what you use, metered per unit such as tokens, requests or audio minutes, with no subscription or minimum commitment. Most LLM APIs bill this way, from a prepaid balance or a periodic invoice.

[Chat Completions API](https://broadrouter.com/glossary/chat-completions-api/)

The Chat Completions API is OpenAI's HTTP endpoint, POST /v1/chat/completions, that takes a model name and a list of role-tagged messages and returns the model's next message. Its request and response shape became the common format that OpenAI-compatible providers and SDKs reuse.

[Concurrency](https://broadrouter.com/glossary/concurrency/)

Concurrency is the number of requests a client has in flight at the same time. LLM APIs often cap it per key or plan, separately from requests per minute, because each open request holds GPU capacity until its response has finished.

[Prepaid credit](https://broadrouter.com/glossary/prepaid-credit/)

Prepaid credit is money added to an API account before use. Each request deducts its cost from the balance, and when the balance runs out, metered requests stop until you top up. It caps spend at what you have paid in.

[Rate limit](https://broadrouter.com/glossary/rate-limit/)

A rate limit is a cap an API places on how much a client can use it in a time window, such as requests per minute or tokens per minute. Requests over the limit are refused, usually with HTTP 429 Too Many Requests, until usage falls back under it.

[Structured output](https://broadrouter.com/glossary/structured-output/)

Structured output is an LLM API feature that constrains a model's reply to a format the developer specifies, usually a JSON Schema, so code can parse it reliably. It is stricter than JSON mode, which guarantees valid JSON but not a particular shape.

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

## Questions

### Why does DSPy call /v1/responses on BroadNet?

DSPy 3.5's dspy.LM reference says model\_type 'auto', the default, makes OpenAI models use the Responses API. An openai/ model with a custom api\_base is treated the same way. Pass model\_type="chat" to use chat completions, which is what BroadNet serves.

### How do I write the BroadNet model ID in DSPy?

Put openai/ in front of the BroadNet ID, for example openai/broadnet/glm-5.3. DSPy uses openai/ to pick the client and, in our test, sent broadnet/glm-5.3 as the model.

### Which BroadNet model suits DSPy optimizers?

Optimizers make many calls, so cost and rate limits matter. DeepSeek V4.1 Flash has a low per-token price and is the model on BroadNet's flat-price plans, which aren't metered per token. Use GLM 5.3 or Qwen3.8 27B for modules that call tools, such as ReAct.

### Does DSPy's cache matter with BroadNet?

DSPy caches LM calls by default (cache=True), so repeating the same call costs nothing. Set cache=False while you are checking the connection, or you may be looking at a cached answer.

## Related guides

-   [Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)
-   [Route LiteLLM to BroadNet Inference (SDK and proxy)](https://broadrouter.com/guides/litellm-proxy-custom-provider/)
-   [Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)
-   [BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/)
-   [All tools that work with BroadNet Inference](https://broadrouter.com/integrations/): SDKs, coding agents, chat UIs and automation tools, with exact settings for each

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-dspy-openai-compatible&utm_content=page-cta) ·

**Sources**

-   [DSPy docs, Installation and language model setup (page dated 2026-11-10, checked 2026-11-15)](https://dspy.ai/current/getting-started/installation/)
-   [DSPy docs, dspy.LM API reference (model\_type) (checked 2026-11-15)](https://dspy.ai/current/api/models/LM/)
-   [PyPI, dspy 3.5.0 (checked 2026-11-15)](https://pypi.org/project/dspy/)
-   [BroadNet Inference API reference (checked 2026-11-16)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-dspy-openai-compatible&utm_content=sources)
