LiteLLM gives one interface to many LLM providers, as a Python SDK and as a proxy server. To reach an OpenAI-compatible API, its docs say to do two things: put openai/ in front of the model name, and set api_base. BroadNet Inference uses that route.
1. Python SDK
pip install litellm
export BROADNET_API_KEY='your-broadnet-key'
import os
import litellm
response = litellm.completion(
model="openai/broadnet/glm-5.3",
api_base="https://inference.broadnet.ai/v1",
api_key=os.environ["BROADNET_API_KEY"],
max_tokens=4000, # reasoning counts toward this
messages=[{"role": "user", "content": "Name three uses of a bloom filter."}],
)
print(response.choices[0].message.content)
openai/ selects the route; LiteLLM sends broadnet/glm-5.3 to BroadNet. Don’t add anything to the base URL; LiteLLM’s OpenAI client adds the endpoint path.
LiteLLM’s docs note that the openai route uses the official OpenAI Python library, which needs an API key on every request. Pass api_key explicitly, as above, so LiteLLM never falls back to an OPENAI_API_KEY meant for OpenAI itself. The response comes back in the OpenAI shape, so the rest of your code doesn’t change.
When to use the proxy
The SDK is enough inside one Python app. The proxy helps when several apps or languages need the same models: they all call one OpenAI-style endpoint with short model names, and the proxy holds the BroadNet key, applies rate limits and logs usage. Your apps then need no BroadNet-specific settings at all.
2. Proxy server
Install the proxy extras and write config.yaml:
pip install 'litellm[proxy]'
model_list:
- model_name: glm
litellm_params:
model: openai/broadnet/glm-5.3
api_base: https://inference.broadnet.ai/v1
api_key: os.environ/BROADNET_API_KEY_GLM
rpm: 30
- model_name: deepseek-flash
litellm_params:
model: openai/broadnet/deepseek-v4.1-flash
api_base: https://inference.broadnet.ai/v1
api_key: os.environ/BROADNET_API_KEY_DEEPSEEK
rpm: 30
- model_name: qwen
litellm_params:
model: openai/broadnet/qwen3.8-27b
api_base: https://inference.broadnet.ai/v1
api_key: os.environ/BROADNET_API_KEY_QWEN
rpm: 30
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
Start it and call it like any OpenAI endpoint:
litellm --config config.yaml
# in a second terminal
curl http://localhost:4000/chat/completions \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "glm", "messages": [{"role": "user", "content": "Hello"}]}'
os.environ/BROADNET_API_KEY_GLM tells LiteLLM to read that key from the environment. Each entry uses its own BroadNet pay-as-you-go key, so rpm: 30, LiteLLM’s optional per-deployment rate limit, matches each key’s limit. If several entries share one key, they share that one limit, so split the number between them. master_key makes the proxy require a key on every call; LiteLLM’s docs say it must start with sk-. Export LITELLM_MASTER_KEY before you start the proxy.
Which model to pick
| model_name | BroadNet model | Use |
|---|---|---|
glm |
GLM 5.3 | Tools, structured output, 1M context |
deepseek-flash |
DeepSeek V4.1 Flash | Long context; the unlimited-plan model |
qwen |
Qwen3.8 27B | Images, tools, Arabic |
| (add if needed) | Gemma 4 VL 26B | Low per-token price; works on the $5 credit |
Prices are on the pricing page. If you are moving from another gateway, the migration checklist covers what to test.
Troubleshooting
- 401. The BroadNet key is wrong, expired, or out of pay-as-you-go allowance. If the 401 comes from the proxy itself, check the LiteLLM key you sent instead.
- Not Found. Make sure
api_baseishttps://inference.broadnet.ai/v1with/v1. If the base URL is right, check the model ID afteropenai/matches BroadNet’s exactly. - Model refused on a new account. GLM 5.3, Qwen3.8 27B and DeepSeek V4.1 Flash need a top-up first; the $5 credit covers only
broadnet/gemma-4-26b-a4band the two Transcribe models. - Empty content. Reasoning counts toward
max_tokens. Raise it or sendreasoning_effort="low". Why answers come back empty. - 429. A pay-as-you-go key allows 30 requests a minute and 4 at once. LiteLLM’s
rpmhelps with the first; keep concurrent callers at 4 or fewer per key. See errors and retries. - Parallel agents on a plan key. One plan session runs one request at a time. Running several agents through the proxy on one session makes them queue; buy one session per parallel agent.
