Structured Output

Pass responseFormat to generateText() or streamText() to request structured responses. The SDK translates the unified shape to the provider's API. Support and schema constraints depend on the chosen model and endpoint. Validate the completed output before using it; refusals and token-limit truncation can still produce a response that does not match your schema.

import { generateText } from "@yourgpt/llm-sdk";
import { openai } from "@yourgpt/llm-sdk/openai";

const result = await generateText({
  model: openai("gpt-4o"),
  prompt: "List the top 3 fastest land animals.",
  responseFormat: {
    type: "json_schema",
    json_schema: {
      name: "animals_response",
      schema: {
        type: "object",
        properties: {
          animals: {
            type: "array",
            items: {
              type: "object",
              properties: {
                name: { type: "string" },
                top_speed_kmh: { type: "number" },
              },
              required: ["name", "top_speed_kmh"],
            },
          },
        },
        required: ["animals"],
      },
      strict: true,
    },
  },
});

const data = JSON.parse(result.text);
// → { animals: [{ name: 'Cheetah', top_speed_kmh: 120 }, ...] }

ResponseFormat shape

The unified type uses OpenAI's response_format shape — callers who already write response_format for OpenAI can pass it through unchanged.

type ResponseFormat =
  | { type: "json_object" }
  | {
      type: "json_schema";
      json_schema: {
        name: string;
        schema: Record<string, unknown>; // JSON Schema
        strict?: boolean; // default: true
      };
    };

• type: 'json_object' — free-form JSON, no schema enforcement. Adapters that don't have a native "JSON mode without schema" (Anthropic) inject a system-prompt suffix asking for JSON instead.
• type: 'json_schema' — schema-validated output. Recommended.

Per-provider translation

Each adapter translates responseFormat to its provider's native field:

You don't need to think about this — the SDK handles it. The notes below matter only if you hit edge cases.

Provider gotchas

Anthropic — schema sanitization

Anthropic's structured-output schema subset is narrower than OpenAI's. The adapter automatically strips keys Anthropic rejects so your call doesn't 400:

• Stripped: minimum, maximum, exclusiveMinimum, exclusiveMaximum, multipleOf, minLength, maxLength, minItems, maxItems, minProperties, maxProperties, pattern, $schema
• Converted: oneOf → anyOf (Anthropic accepts the latter, not the former)
• Forced: additionalProperties: false on every object

If you rely on numeric or length constraints for validation, do that client-side after JSON.parse() rather than encoding it in the schema.

Schema property names are data: a field named minimum, pattern, or oneOf is preserved even when a schema keyword of the same name is removed or translated. The SDK does not mutate your original schema.

When streaming a multi-step tool workflow, accumulated text may include progress text from earlier steps. Parse the final assistant message in result.response after completion, rather than concatenated text from every step. The provider compatibility checks cover tool continuation followed by a schema response.

Google Gemini — OpenAPI subset

Gemini's responseJsonSchema accepts an OpenAPI 3.0 subset. The adapter strips keys Gemini doesn't recognize:

• Stripped: oneOf, anyOf, $ref, $defs, definitions, pattern, $schema, additionalProperties

Schemas with discriminated unions or shared definitions need to be inlined before passing to Gemini.

xAI — additionalProperties default

xAI inverts OpenAI's default: additionalProperties defaults to false and must be explicitly set true if you want extra properties allowed. The adapter passes your schema through unchanged, so be explicit.

Ollama — local only

Ollama's format field requires Ollama v0.5+ for schema-constrained output (string "json" works on older versions for free-form JSON). Ollama Cloud does not support structured outputs at the time of writing.

Capability gate

Each model in the registry carries a supportsJsonMode capability flag. When you pass responseFormat to a model that doesn't support it, the SDK logs a warning:

[llm-sdk] anthropic/claude-3-haiku-20240307 does not support structured
output (responseFormat); the request will be sent but the provider may
ignore it.

This is a warning, not an error — the request still goes through. Switch to a supported model (e.g. claude-3-5-sonnet-latest) or open an issue if you need fallback behavior.

Reasoning models — token semantics

For OpenAI reasoning models (o1, o3, o4, gpt-5.x):

• maxTokens is internally translated to max_completion_tokens
• temperature is silently dropped (these models reject it)
• max_completion_tokens includes BOTH reasoning tokens AND visible output tokens — set generously (maxTokens: 4000+) or you may see truncated responses

const result = await generateText({
  model: openai('o3-mini'),
  prompt: 'Solve: ...',
  maxTokens: 4000,           // → max_completion_tokens internally
  temperature: 0.7,          // → silently dropped
  responseFormat: { ... },
});

Fallback chains

responseFormat works through fallback chains transparently. Each provider in the chain receives the schema in its native format:

import { createFallbackChain } from '@yourgpt/llm-sdk/fallback';
import { createOpenAI } from '@yourgpt/llm-sdk/openai';
import { createAnthropic } from '@yourgpt/llm-sdk/anthropic';

const chain = createFallbackChain({
  models: [
    openai.languageModel('gpt-4o'),
    anthropic.languageModel('claude-3-5-sonnet-latest'),
  ],
  strategy: 'priority',
});

// Same responseFormat works on either hop
const result = await chain.chat({
  messages: [...],
  config: {
    responseFormat: { type: 'json_schema', json_schema: { ... } },
  },
});

A working end-to-end demo lives in examples/fallback-demo — see the /chat/structured route.

Next Steps

• generateText() — full text generation API
• streamText() — streaming variant
• Tools — function calling (orthogonal to structured output)