Structured Outputs & Tool Use

Why it's critical

Free text is great for humans, but code can't work with it. If the model answers "the lead looks solid, about 85, worth following up" — your software doesn't know what the score is. But if it returns {"score": 85, "priority": "high"} — you can feed that straight into a database, a condition, or the next step in a flow.

Structured Outputs are the ability to force the model to return a fixed, valid data structure. Tool Use (or Function Calling) is the next step — the model doesn't just return data, it chooses which function to call and with which parameters. These two are the foundation of any serious automation, agent or integration.

JSON & Structured Output

The basic way: ask for JSON in the prompt and define a schema. Modern models have a dedicated mode that guarantees valid output. Example with OpenAI (Python):

from openai import OpenAI
client = OpenAI()

resp = client.chat.completions.create(
    model="gpt-5.6",
    response_format={"type": "json_schema", "json_schema": {
        "name": "lead_score",
        "schema": {
            "type": "object",
            "properties": {
                "score": {"type": "integer", "minimum": 0, "maximum": 100},
                "priority": {"type": "string", "enum": ["high", "medium", "low"]},
                "reason": {"type": "string"}
            },
            "required": ["score", "priority", "reason"],
            "additionalProperties": False
        }
    }},
    messages=[
        {"role": "system", "content": "Score the lead quality from the details."},
        {"role": "user", "content": "A SaaS company, 50 employees, defined budget, high urgency."}
    ],
)
import json
data = json.loads(resp.choices[0].message.content)
print(data["score"], data["priority"])

The schema guarantees score is a number between 0 and 100 and priority is one of the allowed values. That's far more reliable than "ask for JSON" in the prompt alone.

Tool Use / Function Calling

Here the model decides on its own which tools to call. You define the available tools, and the model returns which one to call and with which parameters. Example with Anthropic (Claude):

import anthropic
client = anthropic.Anthropic()

tools = [{
    "name": "get_weather",
    "description": "Returns the current weather in a given city",
    "input_schema": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
    }
}]

msg = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    tools=tools,
    messages=[{"role": "user", "content": "What's the weather in Tel Aviv?"}],
)

for block in msg.content:
    if block.type == "tool_use":
        print(block.name, block.input)   # get_weather {'city': 'Tel Aviv'}
        # here you run the real function and return the result to the model

The full flow: (1) the model asks to call a tool; (2) your code runs the real function; (3) you return the result to the model; (4) the model composes a final answer. This is exactly the mechanism behind AI agents and MCP.

Validation & Retry — don't trust, verify

Even with json_schema, it's worth validating on your side before feeding the system. In Python people use Pydantic; in TypeScript — Zod.

from pydantic import BaseModel, Field, ValidationError

class LeadScore(BaseModel):
    score: int = Field(ge=0, le=100)
    priority: str
    reason: str

def parse_with_retry(client, messages, tries=2):
    for attempt in range(tries):
        raw = call_model(client, messages)          # call the model
        try:
            return LeadScore.model_validate_json(raw)  # validate
        except ValidationError as e:
            # return the error to the model and ask it to fix
            messages.append({"role": "user",
                "content": f"The output was invalid: {e}. Return valid JSON only."})
    raise RuntimeError("failed to get valid output")

This pattern — try, validate, and if it fails return the error to the model and ask for a fix — is an industry standard. It turns a fragile system into a stable one.

Tips & common mistakes

• Low temperature (0–0.2) for structured outputs — less "creativity," more consistency.
• additionalProperties: false in the schema — prevents the model from adding unexpected fields.
• Clear field names help the model — due_date beats d.
• Don't ask for free text + JSON in the same answer. It breaks parsing. Separate the calls.
• Always wrap in try/except. Even the best model can occasionally return invalid output.
• Limit the number of tools. Too many tools confuse the model — group by context.