The moment a prompt's output needs to be parsed by code instead of read by a person, "please respond in JSON" stops being good enough. Models asked nicely for JSON will happily wrap it in a markdown code fence, add a friendly sentence before it, or drop a trailing comma that breaks every standard parser. Structured output features exist specifically to close that gap.
Why asking nicely isn't enough
A plain instruction to "return JSON" is competing against everything else the model has learned about being a helpful conversational assistant - explaining itself, hedging, formatting for readability. Left to its own devices, a model will often produce something that looks like JSON to a human but fails to parse: unescaped quotes inside a string, a comment slipped in, or prose wrapped around the object. That's the failure mode structured output APIs are built to eliminate.
JSON mode: constraining the output format
Most major model providers now offer a JSON mode - a request parameter that constrains the model's output to be syntactically valid JSON, removing the wrapping prose and formatting inconsistencies almost entirely. It's a meaningful reliability upgrade over a plain instruction, but it only guarantees the output parses as JSON - it does not guarantee the JSON has the specific fields, types, or structure you actually need.
Schema-constrained output: guaranteeing the shape too
Schema-constrained output (sometimes called structured outputs or tool-call-based extraction) goes a step further: you supply an explicit schema - field names, types, which fields are required - and the model's output is constrained to match it, not just to be valid JSON in general. This is the more reliable option whenever you're extracting specific fields for a database, an API call, or any downstream system that expects a fixed structure.
Each step trades a bit of flexibility for a lot more reliability - schema-constrained output is worth the setup cost for anything that feeds a downstream system.
Still validate on your side
Even with schema-constrained output, treat the model's response the way you'd treat any external input: parse it, validate it against your schema in code, and have a defined fallback - retry with an error message fed back to the model, or a default value - for the rare case where something still doesn't match. Structured output features dramatically reduce failures; they don't make validation optional.
When plain prompted JSON is still fine
For quick, low-stakes, one-off scripts where you'll eyeball the output yourself, a plain JSON instruction is often good enough and saves the setup. Reach for JSON mode or schema-constrained output once the output feeds something automated - a pipeline, a database write, another API call - where a malformed response fails silently or loudly breaks something downstream.