Structured JSON Output from LLMs
Unstructured text responses from LLMs are flexible but fragile: parsing them requires regex, string manipulation, or ad-hoc parsing that breaks when the model deviates from expected format. Structured JSON output ensures responses conform to a schema you define, making integration with downstream systems reliable and type-safe. OpenAI's JSON mode guarantees valid JSON; combining it with Pydantic validation gives you end-to-end type checking and error detection.
Using OpenAI's JSON Mode
JSON mode constrains the model to output valid JSON that matches a schema you specify. Enable it by passing response_format parameter:
from openai import OpenAI
import json
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": "You are a JSON extraction assistant. Extract person information from the text and return valid JSON."
},
{
"role": "user",
"content": "Alice is a 28-year-old software engineer from San Francisco. She specializes in Python."
}
],
response_format={"type": "json_object"}
)
# The response is guaranteed to be valid JSON
output = json.loads(response.choices[0].message.content)
print(output)
JSON mode guarantees the response is parseable as JSON, but it does not enforce a schema. The model might return {"name": "Alice"} or {"person": {"name": "Alice"}} depending on what it infers. To enforce structure, provide a more detailed schema or use Pydantic validation.
Defining Schemas with Pydantic
Pydantic models define the exact structure you expect. You can print the JSON schema and pass it to the model, or validate responses against the model:
from openai import OpenAI
from pydantic import BaseModel
import json
client = OpenAI()
# Define the expected structure
class Person(BaseModel):
name: str
age: int
occupation: str
skills: list[str]
# Get the JSON schema from Pydantic
schema = Person.model_json_schema()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": f"""You are a JSON extraction assistant.
Extract person information and return valid JSON matching this schema:
{json.dumps(schema, indent=2)}"""
},
{
"role": "user",
"content": "Alice is a 28-year-old software engineer specializing in Python and Rust."
}
],
response_format={"type": "json_object"}
)
# Parse and validate against the Pydantic model
output_data = json.loads(response.choices[0].message.content)
person = Person(**output_data) # Validates and constructs the object
print(f"Name: {person.name}")
print(f"Age: {person.age}")
print(f"Skills: {', '.join(person.skills)}")
Pydantic validates the model after parsing. If the model returns unexpected fields or missing required fields, Pydantic raises a ValidationError. This catches schema mismatches early.
Handling Lists and Nested Objects
For complex data like multiple people or nested structures, use Pydantic's list and nested model support:
from openai import OpenAI
from pydantic import BaseModel
import json
client = OpenAI()
class Skill(BaseModel):
name: str
years_of_experience: int
class Person(BaseModel):
name: str
age: int
skills: list[Skill] # List of nested objects
class PeopleResponse(BaseModel):
people: list[Person] # List of people
schema = PeopleResponse.model_json_schema()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": f"""Extract all people mentioned and their skills. Return valid JSON.
Schema: {json.dumps(schema, indent=2)}"""
},
{
"role": "user",
"content": """Alice is a 28-year-old developer with 5 years of Python and 2 years of Rust.
Bob is 32 and has 8 years of Java and 3 years of Go."""
}
],
response_format={"type": "json_object"}
)
# Validate and parse
output_data = json.loads(response.choices[0].message.content)
result = PeopleResponse(**output_data)
for person in result.people:
print(f"{person.name} ({person.age})")
for skill in person.skills:
print(f" - {skill.name}: {skill.years_of_experience} years")
Pydantic's nested models allow you to define arbitrarily complex structures. Validation ensures all required fields are present and typed correctly.
Handling Validation Failures Gracefully
Sometimes the model deviates from the schema. Always wrap validation in try-catch:
from openai import OpenAI
from pydantic import BaseModel, ValidationError
import json
client = OpenAI()
class Product(BaseModel):
name: str
price: float
in_stock: bool
schema = Product.model_json_schema()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": f"""Extract product info. Return JSON matching: {json.dumps(schema)}"""
},
{
"role": "user",
"content": "I want to buy a laptop for $1200 and it's in stock."
}
],
response_format={"type": "json_object"}
)
try:
output_data = json.loads(response.choices[0].message.content)
product = Product(**output_data)
print(f"Product: {product.name}, Price: ${product.price}")
except ValidationError as e:
print(f"Validation error: {e}")
print(f"Raw response: {output_data}")
except json.JSONDecodeError as e:
print(f"JSON parse error: {e}")
When validation fails, you can retry with a clearer schema, ask the user for clarification, or fall back to unstructured output. Always log the raw response for debugging.
Union Types and Optional Fields
Pydantic supports optional fields and union types for flexible structures:
from openai import OpenAI
from pydantic import BaseModel, Field
from typing import Optional, Union
import json
client = OpenAI()
class Review(BaseModel):
text: str
rating: int = Field(ge=1, le=5) # Rating between 1 and 5
author: Optional[str] = None # Optional author name
sentiment: Union[str, None] = None # Can be a string or null
schema = Review.model_json_schema()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": f"""Extract review sentiment. Return JSON: {json.dumps(schema)}"""
},
{
"role": "user",
"content": "This book is amazing! Rating: 5"
}
],
response_format={"type": "json_object"}
)
output_data = json.loads(response.choices[0].message.content)
review = Review(**output_data)
print(f"Review: {review.text}")
print(f"Rating: {review.rating}/5")
if review.author:
print(f"Author: {review.author}")
Optional fields (Optional[Type]) allow null or omitted values. Union types enable multiple possible types. Field constraints like Field(ge=1, le=5) enforce ranges.
Performance: Token Cost of Structured Output
Structured output adds overhead in two ways: (1) the schema definition is included in the prompt, consuming tokens, and (2) the model must spend compute enforcing the schema. For large schemas, consider whether structured output is worth the cost, or whether simple regex parsing would suffice for your use case.
If you have many schema definitions, reuse them across related tasks. Define once, reference many times. For performance-critical applications, fine-tune models on your specific schema to reduce per-request overhead.
Key Takeaways
- Enable JSON mode with
response_format={"type": "json_object"}to guarantee valid JSON output. - Use Pydantic models to define and validate exact schema requirements.
- Include the Pydantic schema in system prompts to guide the model toward expected output.
- Always wrap validation in try-catch to handle schema mismatches gracefully.
- Use Pydantic's optional and union types for flexible structures.
- Validate after parsing to catch type mismatches, missing fields, and constraint violations.
Frequently Asked Questions
Does JSON mode always produce valid JSON?
Yes, when enabled, the API guarantees syntactically valid JSON. However, it may not match your intended schema (wrong field names, unexpected fields). Pydantic validation catches semantic mismatches.
Can I enforce a schema without Pydantic?
Yes, by including the schema in the system prompt and using JSON mode. The model will attempt to follow the schema. However, Pydantic provides runtime validation, making failures explicit rather than silent.
What if the model returns extra fields?
By default, Pydantic ignores extra fields. To reject them, set model_config = ConfigDict(extra="forbid") in your Pydantic model. This is strict but ensures the model is not hallucinating fields.
How do I handle optional arrays?
Use Optional[list[Type]] or list[Type] | None:
class Response(BaseModel):
items: Optional[list[str]] = None # Can be list or null
tags: list[str] = [] # Default to empty list if not provided
Can I use dataclasses instead of Pydantic?
Python dataclasses lack schema export, so you would need to manually define JSON schemas. Pydantic is recommended because it generates schemas automatically and validates at runtime.