---
title: "Pydantic AI with a custom OpenAI-compatible base_url"
description: "Run Pydantic AI agents on BroadNet Inference with OpenAIChatModel and OpenAICompatibleProvider, plus tool calls, structured output and model profile fixes."
url: https://broadrouter.com/guides/pydantic-ai-openai-compatible/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Use Pydantic AI with BroadNet Inference (OpenAICompatibleProvider)

Updated 2026-11-16

Install pydantic-ai-slim\[openai\], then build OpenAIChatModel with a BroadNet model ID such as broadnet/glm-5.3 and an OpenAICompatibleProvider whose base URL is https://inference.broadnet.ai/v1 and whose API key is your BroadNet key. Pass the model to Agent. Use OpenAIChatModel, not the Responses model, because BroadNet serves chat completions. Pydantic AI's docs say OpenAIProvider with base\_url is meant for OpenAI itself.

Pydantic AI is a Python agent framework from the Pydantic team. For a service it has no built-in provider for, its docs say to use `OpenAICompatibleProvider` with the service’s `base_url`. BroadNet Inference serves standard chat completions at `https://inference.broadnet.ai/v1`, so that is the route. BroadRouter is BroadNet’s own site; this follows the Pydantic AI docs as of version 2.68.

## Install

```bash
pip install "pydantic-ai-slim[openai]"
export BROADNET_API_KEY='your-broadnet-key'
```

## Connect

```python
import os
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai_compatible import OpenAICompatibleProvider

model = OpenAIChatModel(
    'broadnet/glm-5.3',
    provider=OpenAICompatibleProvider(
        'broadnet',
        base_url='https://inference.broadnet.ai/v1',
        api_key=os.environ['BROADNET_API_KEY'],
    ),
)

agent = Agent(model)
result = agent.run_sync('Give me three names for a Go linting tool.')
print(result.output)
```

Running this is the test: a printed answer means the key, URL and model ID are right.

-   **The first argument is the model ID** sent to BroadNet. Use it exactly, with the `broadnet/` prefix.
-   **The provider name** (`'broadnet'` here) is recorded on every response. Pydantic AI’s docs say to keep it stable, because it decides which data from earlier responses is sent back to the service.
-   **Stay on `OpenAIChatModel`.** Pydantic AI’s docs say to use `OpenAIResponsesModel` only if the service implements the Responses API. BroadNet doesn’t list it.

## Tools and structured output

```python
from pydantic import BaseModel

class City(BaseModel):
    name: str
    country: str

agent = Agent(model, output_type=City)
print(agent.run_sync('Which city hosts the Burj Khalifa?').output)
```

Use `broadnet/glm-5.3` or `broadnet/qwen3.8-27b` for tools and structured output; BroadNet’s docs list both features for these two models only.

## When to set a model profile

Pydantic AI picks a request profile from the model name. If a feature is ignored or rejected, its docs say to pass your own `OpenAIModelProfile`. Two settings are worth knowing for BroadNet:

```python
from pydantic_ai.profiles.openai import OpenAIModelProfile

model = OpenAIChatModel(
    'broadnet/glm-5.3',
    provider=OpenAICompatibleProvider('broadnet', base_url='https://inference.broadnet.ai/v1', api_key=os.environ['BROADNET_API_KEY']),
    profile=OpenAIModelProfile(
        openai_chat_supports_max_completion_tokens=False,  # send max_tokens, the field BroadNet documents
        openai_supports_strict_tool_definition=False,      # only if strict tool schemas are rejected
    ),
)
```

Pass the output limit through model settings, for example `agent.run_sync(prompt, model_settings={'max_tokens': 2000})`. Reasoning counts toward it on DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B.

## Which model to pick

| Agent | Model |
| --- | --- |
| Tools, structured output | `broadnet/glm-5.3` |
| Images or Arabic text, with tools | `broadnet/qwen3.8-27b` |
| Long documents, flat-price plan | `broadnet/deepseek-v4.1-flash` |
| Free starting credit | `broadnet/gemma-4-26b-a4b` (32,768-token context) |

[Prices for every model](https://broadrouter.com/pricing/).

## Troubleshooting

-   **401.** The key is wrong, expired or out of allowance.
-   **404 or model not found.** `base_url` must end in `/v1`. Check the model ID. Paid models need a top-up; the $5 credit covers only `broadnet/gemma-4-26b-a4b` and the two Transcribe models.
-   **Empty output or a validation retry loop.** Reasoning used the output budget. Raise `max_tokens`. [Why answers come back empty](https://broadrouter.com/guides/reasoning-tokens-empty-response/).
-   **400 on tool definitions.** Set `openai_supports_strict_tool_definition=False` in the profile, and use a model BroadNet lists with tools.
-   **429 when you run agents concurrently.** A pay-as-you-go key allows 30 requests a minute and 4 in flight. See [errors and retries](https://broadrouter.com/guides/errors-and-retries/).

## Terms in this guide

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

[Pay-as-you-go](https://broadrouter.com/glossary/pay-as-you-go/)

Pay-as-you-go is a pricing model in which you pay only for what you use, metered per unit such as tokens, requests or audio minutes, with no subscription or minimum commitment. Most LLM APIs bill this way, from a prepaid balance or a periodic invoice.

[Structured output](https://broadrouter.com/glossary/structured-output/)

Structured output is an LLM API feature that constrains a model's reply to a format the developer specifies, usually a JSON Schema, so code can parse it reliably. It is stricter than JSON mode, which guarantees valid JSON but not a particular shape.

[Chat Completions API](https://broadrouter.com/glossary/chat-completions-api/)

The Chat Completions API is OpenAI's HTTP endpoint, POST /v1/chat/completions, that takes a model name and a list of role-tagged messages and returns the model's next message. Its request and response shape became the common format that OpenAI-compatible providers and SDKs reuse.

[Concurrency](https://broadrouter.com/glossary/concurrency/)

Concurrency is the number of requests a client has in flight at the same time. LLM APIs often cap it per key or plan, separately from requests per minute, because each open request holds GPU capacity until its response has finished.

[Prepaid credit](https://broadrouter.com/glossary/prepaid-credit/)

Prepaid credit is money added to an API account before use. Each request deducts its cost from the balance, and when the balance runs out, metered requests stop until you top up. It caps spend at what you have paid in.

[Rate limit](https://broadrouter.com/glossary/rate-limit/)

A rate limit is a cap an API places on how much a client can use it in a time window, such as requests per minute or tokens per minute. Requests over the limit are refused, usually with HTTP 429 Too Many Requests, until usage falls back under it.

## Questions

### Should I use OpenAIProvider or OpenAICompatibleProvider for BroadNet?

OpenAICompatibleProvider. Pydantic AI's docs say OpenAIProvider also takes a base\_url, but it is for OpenAI itself or a transparent proxy to it, and it assumes OpenAI's models and features.

### Why use OpenAIChatModel and not the default OpenAI model?

Pydantic AI's default OpenAI model uses the Responses API. BroadNet serves chat completions at /v1/chat/completions, which is what OpenAIChatModel calls.

### Do tools and structured output work?

On models BroadNet lists with tools and structured output, GLM 5.3 and Qwen3.8 27B, yes. BroadNet's docs don't list tools for DeepSeek V4.1 Flash, so test agents that use tools on it first.

### How do I stop Pydantic AI sending max\_completion\_tokens?

Pass profile=OpenAIModelProfile(openai\_chat\_supports\_max\_completion\_tokens=False) to OpenAIChatModel. Pydantic AI then sends the older max\_tokens field, which is the one BroadNet's API reference describes.

## Related guides

-   [Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)
-   [Use LangChain with BroadNet Inference](https://broadrouter.com/guides/langchain-openai-base-url/)
-   [Use CrewAI with BroadNet Inference as a custom OpenAI endpoint](https://broadrouter.com/guides/crewai-custom-openai-endpoint/)
-   [Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)
-   [All tools that work with BroadNet Inference](https://broadrouter.com/integrations/): SDKs, coding agents, chat UIs and automation tools, with exact settings for each

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-pydantic-ai-openai-compatible&utm_content=page-cta) ·

**Sources**

-   [Pydantic AI docs, Other compatible APIs (Other endpoints, Model profile) (checked 2026-11-15)](https://pydantic.dev/docs/ai/models/compatible-apis/)
-   [Pydantic AI docs, OpenAI models (checked 2026-11-15)](https://pydantic.dev/docs/ai/models/openai/)
-   [PyPI, pydantic-ai 2.68.0, Python 3.11+ (checked 2026-11-15)](https://pypi.org/project/pydantic-ai/)
-   [BroadNet Inference API reference (checked 2026-11-15)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=guides-pydantic-ai-openai-compatible&utm_content=sources)
