# How to Get Reliable JSON From an LLM (Structured Outputs) > Structured outputs make an LLM follow your JSON Schema. How OpenAI, Anthropic and Gemini differ, where they still break, and what to validate. - Canonical: https://zeroraised.com/blog/how-to-get-structured-output-from-an-llm - Published: 2026-11-01T09:20:09Z - Author: ZeroRaised Team (https://zeroraised.com/blog/authors/zeroraised) - Publisher: ZeroRaised (https://zeroraised.com/) - Section: Growth - Tags: artificial-intelligence, developer-tools, saas Getting reliable JSON from an LLM means constraining the model to a schema instead of trusting a prompt that says to reply in JSON. OpenAI's Structured Outputs, Anthropic's structured outputs and Gemini's structured output all target the same outcome: syntactically valid JSON that matches the shape you asked for. None of them removes the need to validate values and handle failures in your own application. **TL;DR** - JSON mode only promises valid JSON, while Structured Outputs promises adherence to your JSON Schema. - OpenAI enables it with `text.format` type `json_schema` and `strict` true, and recommends it over JSON mode. - Anthropic separates JSON outputs from strict tool use, and both use constrained decoding. - Gemini's constrained decoding does not enforce every JSON Schema keyword, so validate values in code. - Refusals and truncated responses can still fall outside the schema, so design for them. **Disclosure:** ZeroRaised publishes this analysis and is a launch site for bootstrapped products. Facts come from the linked sources as checked on 1 Nov 2026; ZeroRaised has no commercial relationship with the companies named. **Structured outputs** is a model feature that constrains generation to a supplied JSON Schema, so required keys appear and enum values stay inside the allowed set ([OpenAI](https://developers.openai.com/api/docs/guides/structured-outputs)). ## What is the difference between JSON mode and structured outputs? OpenAI describes Structured Outputs as the evolution of JSON mode: both produce valid JSON, but only Structured Outputs ensures the response adheres to your schema ([OpenAI](https://developers.openai.com/api/docs/guides/structured-outputs)). That difference matters. A JSON-mode response can be valid JSON and still miss a required key or invent an enum value you never defined. With Structured Outputs, OpenAI says the model will not omit a required key or hallucinate an invalid enum value. The setup differs by one field. Structured Outputs uses `text.format` with type `json_schema` and `strict` set to true, while JSON mode uses type `json_object` ([OpenAI](https://developers.openai.com/api/docs/guides/structured-outputs)). OpenAI recommends Structured Outputs over JSON mode when possible, which is a fair signal that JSON mode is the older, weaker option. ## Should you use tool calling or a structured reply? They solve different problems. OpenAI's guidance is to use function calling when you are connecting the model to tools, functions or data in your system, and a structured `text.format` when you are structuring the reply to the user ([OpenAI](https://developers.openai.com/api/docs/guides/structured-outputs)). Anthropic splits the same idea into two features: JSON outputs through `output_config.format` with type `json_schema`, and strict tool use through `strict: true`, which guarantees schema validation on tool names and inputs ([Anthropic](https://platform.claude.com/docs/en/build-with-claude/structured-outputs)). You can use them separately or together. The split is useful because a support bot that must return a ticket classification needs a different guarantee from an agent that must call the right function with the right arguments. ## How do OpenAI, Anthropic and Gemini compare? OpenAI's SDKs accept Pydantic models in Python and Zod schemas in JavaScript, which keeps the schema next to the types you already use ([OpenAI](https://developers.openai.com/api/docs/guides/structured-outputs)). One practical detail: the first request with a new schema has extra latency, while later requests with the same schema do not. Keep schemas stable rather than generating a new variant for every request. Anthropic uses constrained decoding and makes three claims for it: "Always valid", "Type safe" and "Reliable: No retries needed for schema violations" ([Anthropic](https://platform.claude.com/docs/en/build-with-claude/structured-outputs)). Its JSON Schema support has some limitations, and in the Python SDK you pass a Pydantic model to `client.messages.parse()` as `output_format`. Anthropic also notes that without structured outputs, Claude can produce malformed JSON or invalid tool inputs, including parsing errors, missing fields and inconsistent types. Gemini also uses constrained decoding, but its documentation is explicit that some constraints are not enforced by that engine: `minLength`, `maxLength`, `minProperties`, `maxProperties`, `uniqueItems`, `contains`, `minContains` and `maxContains` ([Gemini API](https://ai.google.dev/gemini-api/docs/structured-output)). The output can be syntactically correct JSON and still violate those rules. Gemini's own instruction is to always validate values in your application. ## Which edge cases still break structured output? Refusals and incomplete responses are the two that OpenAI names ([OpenAI](https://developers.openai.com/api/docs/guides/structured-outputs)). When the model refuses, a `refusal` field appears, and that response may not match your schema. When a response is incomplete, for example because it hit the max output tokens limit, it may also fall outside the schema. Both are expected states, not rare exceptions, and your code should branch on them. Gemini's unenforced keywords are the other category ([Gemini API](https://ai.google.dev/gemini-api/docs/structured-output)). If your product relies on a maximum length or a uniqueness rule, the schema alone will not protect you. The same lesson applies across providers: the schema is a contract about shape, not a proof about values. ## How should you validate and handle failures? Treat the schema as the first check, not the only one. - Validate the parsed object against the same schema in your application, and treat a failure as a handled error rather than a crash. - Distinguish refusal, truncation and validation failure in your logs, because the fixes differ. - Raise the output token limit when truncation is the cause, and do not retry a refusal without changing the request. - Version schemas and store the version with each response, so a later change does not silently break stored data. - Test with hostile inputs, not only the happy path. This is the same discipline as any other model output. The [guide to evaluating LLM outputs](https://zeroraised.com/blog/how-to-evaluate-llm-outputs) covers test sets and scoring, and the [system prompt guide](https://zeroraised.com/blog/how-to-write-a-system-prompt) covers the instruction layer that sits above the schema. ## What this means for bootstrapped founders Structured outputs turn a prompt wish into an engineering contract, and that is the real benefit for a small team. You can build a form-filling feature, a classifier or an extraction pipeline without writing a parser for every malformed response. But do not confuse schema adherence with correctness. The model can return a perfectly shaped object with a wrong value, and no schema keyword will catch that. Budget for validation, logging and a retry policy in the same way you budget for API cost. The [LLM cost guide](https://zeroraised.com/blog/how-to-reduce-llm-costs) and the [LLM API comparison](https://zeroraised.com/blog/how-to-choose-an-llm-api) cover the pricing and vendor side. One more caution: the feature sets differ, so switching providers can change your failure modes. Write the validation layer once, keep it provider-neutral, and you can move without rewriting the product. ## Where ZeroRaised fits ZeroRaised is a launch and discovery site for products built without institutional venture capital or accelerator equity. If you have a product people can try, [submit it here](https://zeroraised.com/submit); the [pricing page](https://zeroraised.com/pricing) explains the current options. For more practical analysis on shipping AI features, [join the ZeroRaised newsletter](https://zeroraised.com/newsletter). No promises of traffic or results, just the useful parts. ## Sources - [OpenAI: Structured model outputs](https://developers.openai.com/api/docs/guides/structured-outputs) - [Anthropic: Structured outputs](https://platform.claude.com/docs/en/build-with-claude/structured-outputs) - [Gemini API: Structured output](https://ai.google.dev/gemini-api/docs/structured-output) ## Frequently asked questions ### What is the difference between JSON mode and structured outputs? Both produce valid JSON, but only structured outputs guarantees that the response adheres to your supplied JSON Schema. JSON mode can return valid JSON that misses a required key or invents an enum value you never defined. ### Do structured outputs remove the need for validation? No. The schema constrains the shape of the response, not the truth of its values. Gemini's documentation also warns that some JSON Schema constraints are not enforced by its decoding engine, so you should validate values in your application. ### When should I use tool calling instead of a structured reply? Use function calling when you are connecting the model to tools, functions or data in your system. Use a structured text.format when you are structuring the reply to the user. Anthropic offers a similar split between JSON outputs and strict tool use. ### Why is my first request with a new schema slower? OpenAI says the first request with a new schema has extra latency, while later requests with the same schema do not. Keep the schema stable rather than generating a new variant on every call. ### Can a refusal break my parser? Yes. OpenAI notes that a refusal adds a refusal field and that the response may not match your schema, and an incomplete response such as one that hits the max output tokens limit may also fall outside it. Branch on those states explicitly.