AI SDK TypeSafe AI 3.0.16 Update: Decision State Gains Image Input Support

The @ai-sdk/typesafe-ai package under the AI SDK umbrella has released version 3.0.16, a patch update focused on a set of low-level enhancements to the experimental decision state system — most notably around multimodal input support and state normalization. While these changes may seem minor on the surface, they have direct implications for developers building AI applications with type-safe patterns.

What's New in This Release

This is a patch release, incrementing from the 3.0.x series to 3.0.16. It was automatically published via GitHub Actions and carries a verified GitHub signature. All changes are contained in a single changeset (9184a35), covering multiple improvements to decision state handling.

The most notable addition is image input support. The update enables image input processing for OpenAI Decisions and language model adapters, meaning decision workflows are no longer limited to plain text — images can now be passed as part of the model's context.

The update also defines clear boundaries: Gateway continues to accept only string and JSON request formats and will reject file-type inputs, and TypeSafe AI itself follows the same policy. In other words, image support is enabled along specific adapter paths, not opened up globally.

Changesets are a widely used versioning strategy for Monorepo projects (powered by @changesets/cli). Developers record their change intent and version impact level (patch / minor / major) alongside their code commits, and CI pipelines use that information to automatically generate changelogs and publish packages. As a multi-package Monorepo, AI SDK uses this mechanism to ensure all version changes across sub-packages are traceable. The changeset ID 9184a35 associated with this release corresponds to a specific change record containing descriptions of all decision state-related modifications. Developers can find the original description file in the .changeset directory of the repository for more detail than what's available in the release notes.

Structured Handling of Decision State

This update introduces "ordered text, file, and JSON parts" into the experimental decision state. This is a structural change — state is no longer a single string or object, but is broken down into a series of ordered "parts."

State Normalization

The update normalizes all public state forms into a parts array before they are passed to decision providers. Text that providers receive is presented as a text part; JSON objects are presented as a json part. This normalization delivers interface consistency: regardless of what form is passed in at the top level, the underlying provider always sees a unified structure.

There is one behavioral change developers should be aware of: arrays passed directly as state are now treated as the parts of the decision state. If you intend to pass a JSON array as data rather than a list of parts, you will need to wrap it in an object or explicitly wrap it as a json part. Otherwise the framework will misinterpret the array elements as state components.

Preserving Native JSON State

When state contains exactly one JSON part, TypeSafe AI preserves the native JSON state form rather than forcing a decomposition. This is an optimization for single-JSON scenarios that avoids unnecessary structural conversion. The update also annotates shared state (shared state) in language model decision prompts, allowing the model to distinguish which portions represent shared context.

Decision state is a core concept in TypeSafe AI's experimental decision system. It describes the contextual information carried when calling an AI model to make a "decision." Unlike ordinary model calls, the decision system aims to abstract the "input context → model judgment → structured output" pipeline into an auditable, traceable state machine under type-safe constraints. In earlier versions, decision state typically existed as a string or a single JSON object — sufficient for pure-text scenarios, but inadequate for complex decision requests that mix multiple data types (textual descriptions + image screenshots + structured parameters). Introducing the ordered parts model is precisely what enables state to carry heterogeneous content — like a message list — while maintaining strong type constraints.

Observability: Improved OpenTelemetry Integration

This update also addresses tracing. Decision file byte data is serialized into OpenTelemetry state attributes using base64 encoding, allowing file content to be captured in telemetry data.

At the span granularity level, the update introduces a layered design: model-call spans record normalized state parts, while outer spans retain the public state inputs. This distinction lets developers see both the raw user-level input and the processed result actually sent to the model when debugging, which is especially useful for troubleshooting multimodal and structured state issues.

OpenTelemetry (OTel) is an open-standard observability framework that provides unified specifications for collecting and exporting tracing, metrics, and logs data. It is widely supported by platforms such as Jaeger, Datadog, and Grafana Tempo. In AI application contexts, OTel's span mechanism can break a single model call into multiple hierarchical trace units, giving developers visibility into latency, inputs/outputs, and errors at each step. Writing file bytes as base64 into span attributes is the standard approach for carrying binary data without modifying the OTel protocol — OTel attribute values only support strings, numbers, and booleans, so binary content must first be serialized to string form. This design allows file content like images to be captured by tracing systems, but it does increase telemetry data volume, which is worth monitoring for storage and transmission costs in high-frequency call scenarios.

Dependency Updates

Alongside this release, several underlying dependencies have also been upgraded:

• @ai-sdk/provider@4.0.25
• @ai-sdk/provider-utils@5.0.57

These coordinated dependency updates indicate that the provider abstraction layer is also adjusting its internal implementation to accommodate the new parts-based and image input models. Projects using the broader AI SDK ecosystem should pay attention to version alignment when upgrading.

What This Means for Developers

Setting aside the version number itself, this update reflects two directions TypeSafe AI is heading with its decision capabilities: first, a move toward multimodality by incorporating images into decision inputs; and second, an evolution toward structured state management, using a unified parts model for text, files, and JSON, with accompanying normalization and observability mechanisms.

For teams already using @ai-sdk/typesafe-ai to build type-safe AI pipelines, the most important thing to check before upgrading to 3.0.16 is any code path that passes arrays directly as state — the behavioral change here could silently alter existing logic. Once confirmed safe, image input support and improved telemetry clarity represent genuine capability gains.