[KongchangAI]
· 2 min read· 1,275 words

Gemini Decision 1 Lands on AI Gateway: The Decision Model That Doesn't Generate Tokens

Gemini Decision 1 Lands on AI Gateway: The Decision Model That Doesn't Generate Tokens

Google launches Gemini Decision 1, a multimodal model built for judgment rather than generation, now available on AI Gateway.

Google's Gemini Decision 1 is now officially available on AI Gateway. Rather than generating content, it makes judgments—returning boolean probabilities, choices, and scores without consuming output tokens. It supports text, image, audio, video, and PDF inputs, making it suitable for high-frequency judgment tasks like request routing, policy checks, agent tool filtering, and result ranking. Calls use the experimental_decide method in AI SDK, with questions declared as typed structured definitions and outputs constrained at the API level. Note that audio and video are only supported via AI SDK 7. Decision calls also count toward logs and budgets. The article argues that separating judgment from generation is a sensible division of labor, but the model provides no explanations, so an auditable chain of reasoning still requires a generative model to fill the gap.

AI Gateway has welcomed a rather different kind of model. Gemini Decision 1 from Google is now officially available, and its role isn't to "generate content" but to "make judgments"—returning boolean probabilities, choices, and scores without consuming output tokens.

Gemini Decision 1 now available on AI Gateway

A multimodal model that doesn't generate

Traditional LLM calls center on "output": you provide a prompt, and it returns text, code, or structured data, with token consumption tied directly to output length. Gemini Decision 1 takes a different path.

According to the official AI Gateway documentation, this multimodal decision model supports text, image, audio, video, and PDF document inputs, and outputs boolean probabilities, choices, and scores. In other words, it doesn't answer "what should be said" but "yes or no," "which one," and "what score."

This design has a direct consequence: calling it incurs no output token overhead. For scenarios that require frequent judgments within a pipeline, this is a structural cost difference—not just a simple parameter optimization.

"Output tokens" are the core unit of LLM billing and performance: every token the model generates requires a forward computation pass and consumes memory bandwidth, so the longer the output, the higher the latency and cost. For classification and judgment tasks that need only one or two bits of information, having a generative model output full text (even if constrained by prompts to a single word) still goes through an autoregressive decoding loop, costing roughly the same as actually generating a paragraph. The decision model's approach limits the output space directly to a finite set of answers—booleans, a few choices, or a score—thereby bypassing per-token decoding, which is the technical root of its "no output token consumption." Understanding this helps determine which tasks are worth migrating to this kind of model.

Typical scenarios: routing, policy, and tool filtering

The official list of use cases points to the most mundane yet resource-intensive parts of agent engineering.

Request routing is the most intuitive category. When a system receives user input, it often needs to first determine which branch the intent belongs to before handing it off to the corresponding processing logic. Using a generative model for this is like using a cannon to kill a mosquito—not only do you need complex prompts to constrain the format, but you also pay output tokens for every classification. The decision model compresses this into a single boolean or choice judgment.

Policy checks apply equally well. For example, reviewing whether a request violates rules or determining whether an operation requires human intervention is essentially a binary or multi-class classification problem, where probabilities and scores are more aligned with what downstream programs need than a natural language explanation.

Filtering agent tools is another high-frequency scenario. When an agent faces dozens of available tools, trying each one is costly; running a first-pass filter with the decision model can significantly narrow the candidate set. Result ranking is also on the list—scoring and ranking candidate results is more efficient than having a generative model explain the pros and cons of each one.

What the code looks like

The example from AI Gateway is quite illustrative. When calling via AI SDK, you use the experimental_decide method, passing the model identifier google/gemini-decision-1, the current state text, and a "question definition" object:

import { experimental_decide as decide } from 'ai';

const result = await decide({
  model: 'google/gemini-decision-1',
  state: 'I was charged twice. Please refund the duplicate charge.',
  questions: {
    requestsRefund: {
      type: 'boolean',
      instructions: 'Does the customer request a refund?',
    },
  },
});

console.log(result.answers.requestsRefund);

You may not have noticed the structure of the question definition: each question has a type (boolean, choice, score) and instructions, and the model returns the corresponding answer field as declared. This is far more stable than having a generative model output JSON and then parsing it, because the output structure is constrained at the API level rather than relying on prompt-based hoping.

The AI SDK mentioned here is an open-source TypeScript toolkit from Vercel for calling models from different vendors through a unified interface; the experimental_ prefix indicates that the API is still experimental and its signature may change. The "question definition" object essentially declares the output schema to the API upfront: the model no longer generates text freely but returns fields under given type constraints, and the server validates and parses accordingly. This is fundamentally different from the traditional approach of "having the model output JSON and then extracting it with regex or a parser"—the latter relies on the model following prompts and requires retries on failure; the former guarantees structural validity at the interface layer, representing a further tightening of the structured output approach.

Differences in multimodal input support across access methods

An easily overlooked detail is that different access methods do not support input modalities consistently.

PDF, audio, and video—the three heavier inputs—are currently supported only through AI SDK 7. The TypeSafe-compatible API and the Decisions API support only text and image inputs. This means that if your business relies on audio or video for judgments, your choice of access path will be constrained and needs to be confirmed in advance.

In terms of invocation, AI Gateway offers three paths: AI SDK, the TypeSafe-compatible API, and the Decisions API. For teams already using AI Gateway, the migration cost mainly involves changing the calling method rather than restructuring the entire data flow.

AI Gateway is a unified proxy layer between applications and various model providers, handling authentication, routing, usage tracking, and logging, so teams don't need to integrate SDKs and billing systems separately for each model. It typically offers two interface forms: one is native invocation for client libraries like AI SDK, and the other is a REST interface aligned with OpenAI-style compatible APIs, making it easy for existing code to switch at low cost. The inconsistency in supported input modalities across access paths often stems from differences in request body specifications and file upload mechanisms across interfaces, so when dealing with large files such as audio and video, you need to first confirm whether the chosen path has the corresponding transmission and parsing capabilities.

Integration with AI Gateway billing and logging

Decision calls appear in logs and count toward budgets just like other model requests. This is critical for production environments: although the decision model doesn't produce output tokens, the call itself is still a tracked and metered resource consumption, and teams can see its share in a unified usage view.

AI Gateway's positioning is "unified access to models, tracking usage and costs," and Decision 1's integration continues this approach. For teams using multiple models simultaneously, bringing decision-type calls into the same observability system is more convenient than maintaining a separate classification service.

What this kind of model means

Separating "judgment" from "generation" is a direction worth watching. The bottleneck of agent systems is often not generation quality but the latency and cost of each decision step. Using a model specifically optimized for branch decisions and letting generative models focus on the parts that truly require creativity—this division of labor is becoming increasingly reasonable from an engineering perspective.

Of course, the boundaries of the decision model are also clear: it outputs probabilities, choices, and scores, without providing explanations. When the business requires an auditable chain of reasoning, a generative model or an additional explanation layer is still needed to fill the gap. For purely programmatic consumption scenarios, this trade-off is worthwhile.

Developers who want to get started can consult the AI Gateway decision documentation to learn how to use the three question types—Choice, Score, and Boolean—or browse the available models list to pick a suitable decision model.

#Gemini Decision 1#AI Gateway#decision model#multimodal model#no output tokens#AI SDK#request routing#agent tool filtering
Share:

Related articles