---
title: "JSON mode vs JSON schema structured output, with code"
description: "JSON mode gives valid JSON; json_schema gives your shape. How to request each on OpenAI-compatible models, validate with Pydantic, and repair bad output in a loop."
url: https://broadrouter.com/blog/structured-output-json-schema/
lang: en
updated: 2026-11-14
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# JSON mode vs json\_schema: structured output that holds up

By the BroadNet Inference team · 4 min read · Published 2026-11-14

JSON mode (response\_format json\_object) only promises valid JSON. Structured output (response\_format json\_schema) also promises your fields and types, where the provider enforces it. Either way, check finish\_reason, parse, and validate against a schema such as a Pydantic model. If validation fails, send the error back and ask again, with a hard limit on attempts.

## Key takeaways

-   OpenAI's guide: both modes produce valid JSON, but "only Structured Outputs ensure schema adherence".
-   Support varies across OpenAI-compatible providers. Test json\_schema first, fall back to json\_object, then to prompt-only.
-   Validate every reply. A reply cut off by max\_tokens is broken JSON whatever the schema says.
-   Repair loops work when you send the exact validation error back. Cap them at two or three attempts.
-   On BroadNet Inference, GLM 5.3 and Qwen3.8 27B list structured output.

**On this page**

1.  What is the difference between JSON mode and json\_schema?
2.  How do you request json\_schema?
3.  Which mode does your provider support?
4.  How do you validate and repair bad output?
5.  What schema features cause trouble?
6.  How do you monitor structured output in production?
7.  When should you use a tool instead?
8.  Where BroadNet Inference fits

Your code needs data, and models write text. [Structured output](https://broadrouter.com/glossary/structured-output/) closes that gap, but “the model returns JSON” means three different things depending on how you ask. This post compares the options on OpenAI-compatible open models, shows how to request each, and gives a validate-and-repair loop you can drop into production.

## What is the difference between JSON mode and json\_schema?

| Method | Request | What you’re promised |
| --- | --- | --- |
| Prompt only | “Reply in JSON with fields…” | Nothing; usually JSON, sometimes with extra text |
| JSON mode | `response_format: {"type": "json_object"}` | Valid JSON, any shape |
| Structured output | `response_format: {"type": "json_schema", ...}` | JSON matching your schema, where enforced |

OpenAI’s Structured Outputs guide puts it directly: “While both ensure valid JSON is produced, only Structured Outputs ensure schema adherence.” It recommends structured output over JSON mode whenever possible.

On open-model providers, the guarantee depends on how the server implements it. Many use constrained decoding, which blocks tokens that would break the schema. Some accept the field but don’t enforce it. That’s why the code below validates regardless.

## How do you request json\_schema?

```python
import os
from openai import OpenAI
from pydantic import BaseModel, Field

client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"])

class Invoice(BaseModel):
    vendor: str
    currency: str = Field(pattern="^[A-Z]{3}$")
    total: float
    line_items: list[float]

schema = Invoice.model_json_schema()
schema["additionalProperties"] = False

response_format = {"type": "json_schema",
                   "json_schema": {"name": "invoice", "strict": True, "schema": schema}}
```

Pydantic generates the JSON Schema from your class, so the schema and the validator never drift apart. If you don’t use Pydantic, the [JSON schema generator](https://broadrouter.com/tools/json-schema-generator/) builds a strict-mode schema from a sample object. For strict mode, OpenAI requires `additionalProperties: false` and every property listed as required.

## Which mode does your provider support?

Run this test once per model before you build on it:

1.  Send `json_schema` with a prompt that invites an extra field (“also add a short comment”).
2.  If the request errors, the provider doesn’t accept `json_schema`. Try `json_object`.
3.  If the reply contains the extra field, the schema isn’t enforced. Treat it as JSON mode.
4.  If `json_object` also errors, use prompt-only with validation.

On BroadNet Inference, the docs list structured output for [GLM 5.3](https://broadrouter.com/models/glm-5.3/) and [Qwen3.8 27B](https://broadrouter.com/models/qwen3.8-27b/) ([compared side by side](https://broadrouter.com/compare/glm-5.3-vs-qwen3.8-27b/)), but don’t name the request parameter. Run the test above on the model you ship.

## How do you validate and repair bad output?

```python
import json
from pydantic import ValidationError

def extract(text: str, model="broadnet/glm-5.3", attempts=3) -> Invoice:
    messages = [
        {"role": "system", "content": "Extract the invoice. Reply with JSON only."},
        {"role": "user", "content": text},
    ]
    for attempt in range(attempts):
        r = client.chat.completions.create(model=model, messages=messages,
                                           response_format=response_format,
                                           max_tokens=4000)
        choice = r.choices[0]
        if choice.finish_reason == "length":
            raise RuntimeError("Cut off: raise max_tokens")  # retrying won't help
        if getattr(choice.message, "refusal", None):
            raise RuntimeError(f"Refused: {choice.message.refusal}")
        raw = choice.message.content or ""
        try:
            inv = Invoice.model_validate_json(raw)
            if abs(sum(inv.line_items) - inv.total) > 0.01:
                raise ValueError("total must equal the sum of line_items")
            return inv
        except (ValidationError, ValueError) as e:
            messages += [
                {"role": "assistant", "content": raw},
                {"role": "user", "content": f"That JSON is invalid: {e}. Return corrected JSON only."},
            ]
    raise RuntimeError(f"No valid JSON after {attempts} attempts")
```

What makes this loop work:

-   **Check `finish_reason` first.** A reply that hit [`max_tokens`](https://broadrouter.com/glossary/max-tokens/) is truncated JSON. Retrying with the same limit fails the same way. Both models above reason before answering, and [reasoning counts toward the limit](https://broadrouter.com/guides/reasoning-tokens-empty-response/), so leave room.
-   **Handle refusals separately.** OpenAI’s guide notes a refusal “does not necessarily follow the schema you have supplied”.
-   **Send the exact error back.** A blind retry often returns the same mistake. “line\_items: field required” gives the model something to fix.
-   **Check business rules in code.** A schema can’t say a total must equal its line items. The check above can.
-   **Cap attempts.** Every attempt resends the whole prompt as [input tokens](https://broadrouter.com/glossary/input-and-output-tokens/).

## What schema features cause trouble?

Keep schemas boring:

1.  **Flat beats nested.** Two levels is plenty for most extraction.
2.  **Enums over free strings** for categories.
3.  **No open-ended objects.** Free-form `dict` fields defeat the point.
4.  **Short field names with descriptions.** Descriptions in the schema guide the model.
5.  **Optional fields as nullable.** In strict mode, list every field as required and allow `null`.

OpenAI’s guide notes some JSON Schema features are unsupported in strict mode. Open-model servers have their own limits. If a schema is rejected, simplify it before you switch modes.

## How do you monitor structured output in production?

A loop that repairs silently can hide a problem that costs you tokens every day. Log three numbers per call site:

| Metric | What it tells you |
| --- | --- |
| First-attempt success rate | How well the model and schema fit; a drop often follows a prompt or model change |
| Average attempts per success | The token cost of repairs |
| Final failures | Inputs your pipeline can’t handle; review them by hand |

If first-attempt success falls below what you’re used to, simplify the schema or try another model before you add more retries. Keep a sample of failed raw replies, so you can test fixes against real cases.

## When should you use a tool instead?

Use structured output for the **final answer**: an extraction, a classification, a plan. Use [tool calling](https://broadrouter.com/blog/tool-calling-open-models/) when the model needs your code to **do** something and continue. Many agents use both: tools during the loop, a schema for the last step. For an image-to-JSON example, see [reading Arabic invoices with a vision model](https://broadrouter.com/guides/read-arabic-invoices-with-vision/).

## Where BroadNet Inference fits

BroadNet Inference lists structured output for GLM 5.3 and Qwen3.8 27B, behind the standard chat-completions format, so the code above runs with the [OpenAI Python SDK](https://broadrouter.com/guides/openai-python-sdk/) unchanged. Qwen3.8 27B also takes images, which suits document extraction. BroadNet’s docs don’t state structured output for DeepSeek V4.1 Flash or Gemma 4 VL 26B, so use the prompt-only path with validation there. Compare per-token prices on the [pricing page](https://broadrouter.com/pricing/).

Free tools for this

-   [OpenAI to BroadNet migration helper](https://broadrouter.com/tools/openai-to-broadnet-migration/) Paste OpenAI SDK code; get base URL, key and model edits, a diff and the checks that remain.
-   [max\_tokens calculator for reasoning models](https://broadrouter.com/tools/reasoning-token-budget/) Answer length + reasoning budget + margin = a max\_tokens that does not cut the answer off.
-   [JSON schema generator for LLM structured output](https://broadrouter.com/tools/json-schema-generator/) Sample JSON to strict schema, Pydantic, Zod and a ready request.
-   [LLM JSON output validator](https://broadrouter.com/tools/llm-json-output-validator/) Parse errors by line and column, safe repairs, cut-off detection and JSON Schema checks for model output.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[Structured output](https://broadrouter.com/glossary/structured-output/)

Structured output is an LLM API feature that constrains a model's reply to a format the developer specifies, usually a JSON Schema, so code can parse it reliably. It is stricter than JSON mode, which guarantees valid JSON but not a particular shape.

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[JSON Schema](https://broadrouter.com/glossary/json-schema/)

JSON Schema is a vocabulary, written in JSON, for describing and validating the structure of JSON data: types, required fields, allowed values and nesting. The current version is 2020-12. LLM APIs use it to define tool parameters and the exact shape a structured output must follow.

[JSON mode](https://broadrouter.com/glossary/json-mode/)

JSON mode is an LLM API setting, response\_format {"type": "json\_object"} in OpenAI-style APIs, that makes the model's reply valid, parseable JSON. Unlike structured outputs with a JSON Schema, it does not guarantee which keys, types or values the object contains.

[Tool calling](https://broadrouter.com/glossary/tool-calling/)

Tool calling, also called function calling, is an LLM API feature in which the model returns a structured request to run a function the developer has defined, with JSON arguments. The application runs the function and sends the result back; the model never runs code itself.

[Input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/)

Input tokens are the tokens a language model reads in a request, including the system prompt, conversation history, tool definitions and images. Output tokens are the tokens it generates in reply, including any reasoning. LLM APIs usually price them separately, and output usually costs more.

[Open-weight model](https://broadrouter.com/glossary/open-weight-model/)

An open-weight model is a machine learning model whose trained weights are published for anyone to download and run, usually under a licence with conditions. It differs from fully open-source AI, which also releases the training code and detailed information about the training data, so others can study and reproduce the model.

[Vision-language model](https://broadrouter.com/glossary/vision-language-model/)

A vision-language model (VLM) is a model that takes images and text together as input and produces text, so it can describe pictures, answer questions about them and read documents, charts or screenshots. VLMs are also called image-text-to-text or multimodal models.

## Questions

### What is the difference between JSON mode and structured output?

JSON mode, response\_format type json\_object, only promises the reply parses as JSON. Structured output, type json\_schema, also promises the fields and types you defined, where the provider enforces it. OpenAI recommends structured output over JSON mode when possible.

### Do open models support json\_schema response\_format?

It depends on the provider. Some accept json\_schema, some only json\_object, and some ignore the field. Send a test request with a schema and a prompt that tempts the model to add an extra field, then check the reply.

### Do I still need to validate structured output?

Yes. Replies can be cut off by max\_tokens, can be refusals, and can break business rules a schema can't express, such as totals that must add up.

### How many times should I retry invalid JSON?

Two or three attempts with the validation error included is usually enough. After that, log the input and route it to a fallback, a stronger model or a person. Unbounded loops waste tokens.

### Should I use structured output or tool calling?

Use structured output when you want the final answer in a fixed shape. Use tool calling when the model needs your code to do something and then continue.

## Keep reading

-   [Arabic text classification with an LLM: sentiment, intent, triage](https://broadrouter.com/blog/arabic-text-classification-llm/)
-   [Tool calling with open models: schemas, parallel calls, retries](https://broadrouter.com/blog/tool-calling-open-models/)
-   [Switching to an OpenAI-compatible API: a migration checklist](https://broadrouter.com/blog/openai-compatible-api-migration-checklist/)
-   [n8n AI Agent on a cheap LLM: setup, model choice and cost per run](https://broadrouter.com/blog/n8n-ai-agent-cheap-llm/)
-   [How to fix invalid JSON from an LLM: causes, repair, retry](https://broadrouter.com/blog/fix-invalid-json-from-llm/)
-   [Arabic named entity extraction with an LLM: JSON, dates and checks](https://broadrouter.com/blog/arabic-named-entity-extraction-llm/)
-   [A private endpoint starts with a clear workload](https://broadrouter.com/blog/private-endpoint-planning/)
-   [Before fine-tuning: define the problem to solve](https://broadrouter.com/blog/fine-tuning-readiness/)
-   [Guide: Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/)
-   [Guide: Read Arabic invoices and receipts with a vision model](https://broadrouter.com/guides/read-arabic-invoices-with-vision/)
-   [Guide: Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-structured-output-json-schema&utm_content=page-cta) ·

**Sources**

-   [OpenAI, Structured Outputs guide (checked 2026-11-14)](https://developers.openai.com/api/docs/guides/structured-outputs)
-   [OpenAI API reference, create chat completion (response\_format)](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create)
-   [Pydantic docs, JSON Schema and validation](https://docs.pydantic.dev/latest/concepts/json_schema/)
-   [BroadNet Inference API reference, models](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-structured-output-json-schema&utm_content=sources#models)
