Why Your Zod .refine() Never Reaches the Model: The Hidden Trap in LLM Structured Output

Hand a Zod 4 schema straight to the OpenAI Node SDK or Vercel AI SDK's structured output helpers, and every .refine() you wrote will be silently dropped before it ever reaches the model. What the model sees isn't the schema you wrote—it's a subset produced by z.toJSONSchema(). And the rules lost along that chain are often exactly the constraints that matter most to your business logic.

A Branch That Gets Skipped

The problem lives in Zod's JSON Schema converter. According to the author's analysis of the Zod v4.6.7 source (released October 2026), a single line in json-schema-processors.ts determines which validation rules make it into the schema:

for (const ch of list) contributors[ch._zod.def.check]?.(agg, ch._zod.def);

The contributors map has only 13 keys: greater_than, less_than, multiple_of, min_length, string_format, and so on. Every .refine() you write is stored internally by Zod as a check named custom. custom isn't in the key list, so the optional call finds no handler and the rule gets skipped outright.

Both the OpenAI Node SDK and the Vercel AI SDK call z.toJSONSchema(), and that JSON is what actually constrains the model. .refine() still runs during response parsing—the model just never received the rule.

What is JSON Schema

JSON Schema is a specification, written in JSON, that describes the structure and constraints of JSON data. It's standardized by the IETF as a series of draft documents (the draft-7 used by the SDK in this article is one such version). Its expressive power has limits: it can describe "structure plus simple value constraints"—types, enums, regex pattern, date format, numeric bounds, required fields, array lengths. But arbitrary custom functions (e.g., "the note must not contain URLs"), cross-field comparisons (e.g., "the refund date must not be earlier than the order date"), and transformations that change data shape (e.g., converting a string to a number) all fall outside its reach. This is precisely why JSON Schema is typically supported by each model API as a subset: OpenAI, Anthropic, and others accept only some of the keywords, and the rest are either ignored or downgraded to natural-language descriptions. Understanding this boundary is the premise for the pain point in this article—Zod's .refine() is fundamentally an arbitrary TypeScript function, and by nature cannot be serialized into JSON Schema.

A Side-by-Side Look at a Refund Extraction Schema

The author demonstrates with a refund information extraction schema containing four kinds of rules: field-level regex, positive(), z.enum, ISO date format, plus .refine() on note, an object-level .refine() across fields, and .transform() on sku.

Comparing the output of z.toJSONSchema(Refund, { io: "input" }) line by line against the Zod source yields the following:

Field	Zod rule	What the model sees
orderId	.regex(...)	pattern
amount	.positive()	exclusiveMinimum: 0
currency	z.enum	enum
orderDate/refundDate	z.iso.date()	format: "date" plus pattern
note	.refine(no links)	only "type": "string" remains
sku	.transform(...)	plain string in input mode; errors in output mode
Entire object	.refine(refund date after order date)	nothing at all

Zod's test files corroborate the note row: when two .refine()-annotated strings are unioned, the snapshot collapses both branches into a bare "type": "string"—that deduplication is only possible if neither refine made it into the output. Same story for object-level refine; .superRefine() and object-level .refine() also land as custom checks.

Another detail comes from the Zod JSON Schema docs: in input mode the object doesn't set additionalProperties—the AI SDK restores it with its own addAdditionalPropertiesToJsonSchema; in default output mode Zod writes additionalProperties: false itself.

z.toJSONSchema() and the io option

z.toJSONSchema(schema, options) is the conversion entry point provided by Zod 4. It compiles a Zod schema into a standard JSON Schema for consumption by a model's structured output interface. The most critical option is io. A Zod schema is inherently bidirectional: the input to z.string().transform(Number) is a string, the output is a number—they're not the same. io: "input" asks the generated schema to describe "the shape before entering Zod"; io: "output" (the default) describes "the shape after Zod parses it." This distinction directly determines which side of a transform the converter takes, and whether additionalProperties: false is written onto objects. In the article, sku behaves differently in the two modes, and the OpenAI helper defaults to not passing io and errors in output mode—the root cause is right here. So when you hand a schema to the model, you need to first decide: what the model produces is the raw input, which means you should be using the io: "input" variant.

The Two Fates of transform: Error Out or Lie

.transform()'s behavior depends on the io option. A transform on a string is essentially a pipe, and the processor picks a side with ctx.io === "input" ? (inIsTransform ? def.out : def.in) : def.out.

In input mode it takes the string side, the model sees { "type": "string" }, and the conversion happens after the data comes back. For sku, a case conversion is harmless. But swap in z.string().transform(Number) and the nature changes: the model is only told "this is a string," so it can confidently write "twelve" or "12 EUR", and Number() quietly hands your business code a NaN—with no validation error whatsoever.

In output mode (the default), the converter runs straight into the transform itself and calls handleUnrepresentable, reporting "Transforms cannot be represented in JSON Schema." The docs say Zod throws unless you pass unrepresentable: "any". The OpenAI helper calls z4.toJSONSchema(schema, { target: 'draft-7', ... }) without touching io or unrepresentable, so it fails during format construction—earlier and louder than silently passing the wrong thing.

How Structured Outputs work

Structured Outputs is a capability offered by OpenAI and others: the caller attaches a JSON Schema to the request, and the server enforces that the model's output satisfies that schema at the decoding stage, rather than just validating after the fact. Implementation typically uses constrained / grammar-based decoding, compiling the schema into a set of grammar rules that mask out any candidate token that would cause a violation, so the model is "physically unable" to emit a non-conforming field name, type, or enum value. This also explains its limits: any constraint that can be compiled into grammar rules (types, required, enums, pattern, etc.) takes effect at decode time; any constraint that requires arbitrary computation or cross-field reasoning cannot be compiled into grammar, and vendors can only choose to ignore it, error out, or—like Anthropic—move it into the field's description text and hope the model "reads it and complies"—which is a retreat from a hard constraint to a soft prompt.

Vendors Peel Off Another Layer

Getting a rule into JSON Schema doesn't mean the model is bound by it—each API supports only a subset.

Anthropic's structured output docs list minimum, maximum, multipleOf, as well as minLength and maxLength, as unsupported. Their SDK helper strips these constraints out, appends them to the field description text, and then validates the response against the original schema. The exclusiveMinimum produced by .positive() belongs to the same class of numeric constraints, so amount > 0 is very likely passed to Claude as description text rather than a decode-time constraint.

OpenAI supports pattern, format, numeric bounds, and anyOf; it lists allOf, not, and if/then/else as unsupported, and requires every field to be in required, with optional fields written as unions with null. Unions have another trap: Zod's union processor emits oneOf for discriminated unions, and the OpenAI helper's override callback rewrites it to anyOf before sending; if you call z.toJSONSchema() yourself and send it to the API directly, none of those patches apply.

Usage: Put Rules in Descriptions, Keep Validation Local

Since refine can't become a representable JSON Schema keyword, the work splits into two places: tell the model the rules in natural language, then catch the response where it still violates them.

The first step is .describe(). Zod copies metadata into the generated JSON Schema, so the description travels to the model alongside the field. Note that it should go at the end of the chain—metadata belongs to a specific schema instance, and every method call returns a new instance, so calling .meta() after .refine() gives you undefined.

export const Refund = z
  .object({
    orderId: z.string().regex(/^ORD-\d{6}$/).describe("Order id, e.g. ORD-123456"),
    amount: z.number().positive().describe("Greater than 0"),
    currency: z.enum(["EUR", "USD"]),
    orderDate: z.iso.date(),
    refundDate: z.iso.date().describe("ISO date, on or after orderDate"),
    note: z
      .string()
      .refine((s) => !/https?:\/\//.test(s), { message: "note must not contain links" })
      .describe("Plain text. No URLs."),
    sku: z.string().describe("SKU exactly as written"),
  })
  .refine((r) => r.refundDate >= r.orderDate, {
    message: "refundDate is before orderDate",
    path: ["refundDate"],
  });

The refines are still there, but they've gone from the only line of defense to a second one. Transforms, meanwhile, should be moved out of the schema sent to the model and used to reshape data after parsing.

The second step is to validate the response yourself and send Zod errors back to the model for one retry. The OpenAI SDK's responses.parse does call the schema's parse, but it throws outright when that fails; switch to responses.create plus safeParse and you get an error object you can send back.

export async function extract<T extends z.ZodType>(
  schema: T,
  name: string,
  prompt: string,
): Promise<z.infer<T>> {
  const input: Msg[] = [{ role: "user", content: prompt }];
  for (let attempt = 0; attempt < 2; attempt++) {
    const res = await openai.responses.create({
      model: "gpt-6-luna",
      input,
      text: { format: zodTextFormat(schema, name) },
    });
    const raw = res.output_text;
    const result = schema.safeParse(JSON.parse(raw));
    if (result.success) return result.data;
    input.push(
      { role: "assistant", content: raw },
      {
        role: "user",
        content:
          "That JSON failed validation:\n" +
          z.prettifyError(result.error) +
          "\nReturn the corrected JSON only.",
      },
    );
  }
  throw new Error(`${name}: failed validation twice`);
}

z.prettifyError() turns each issue into a line prefixed with ✖ and appends → at <path>; the path you set in .refine() determines whether the field name appears in that message, so object-level rules should always set a path.

Keep retries to one: if the model still violates the same rule twice in a row even with the error message in hand, a third attempt is mostly just cost. Throw, log the raw output, then step back and decide whether that rule belongs in the prompt or in your code. Also note that JSON.parse will throw on truncated or refusal responses, so in production you should check the response status before parsing.

.describe() and how metadata propagates

.describe(text) attaches a human-readable explanation to the current schema instance, stored in Zod's metadata. The key point is that the vast majority of Zod's methods are immutable—each call returns a brand new instance: what you get after .refine() is no longer the object you called .describe() on, so the metadata "loses track," and the description from the earlier chain can't be recovered. This is why the article emphasizes putting .describe() at the end of the chain. And the reason it's worth including in the schema sent to the model is that the converter copies metadata into the generated JSON Schema, and most vendors' APIs pass the description field through to the model; so rules that can't be expressed as keywords (no links allowed, date ordering) travel to the model as natural language alongside the field, becoming an extra layer of constraint beyond the prompt. It's a soft prompt, not a guarantee, which is why you still need local .safeParse() as a backstop.

Conclusion: What the Schema Says Isn't What the Model Sees

On the road from demo to a feature people actually depend on, the gap between "the schema I wrote" and "the schema the model sees" is one of the first problems that bites. This isn't a bug in Zod or any particular SDK—it's an inherent boundary between JSON Schema's expressive power, each API's supported subset, and Zod's validation semantics. Putting human-readable rules into descriptions and keeping machine-trustworthy validation after parsing is the more durable division of labor along this chain.

One-line Summary

Zod's .refine() is silently dropped when converting to JSON Schema—the model never receives these constraints.

Paragraph Summary

The article points out that in Zod 4, every .refine() is stored internally as a check named custom, while the contributors map of z.toJSONSchema() covers only 13 built-in validation keys and does not include custom, causing refine rules to be silently dropped. Both the OpenAI Node SDK and Vercel AI SDK rely on this JSON Schema to constrain the model, so field-level and object-level refines are all rendered ineffective, while .transform() either shows the model only a string type (e.g., transform(Number) letting NaN pass silently) or fails outright with an unrepresentable error. Worse, vendors peel off another layer: Anthropic downgrades numeric and length constraints to description text, and OpenAI doesn't support allOf/not/if-then-else. The pragmatic approach is to put rules into .describe(), demote refine to a local second line of defense, and use safeParse to send back errors for one retry.