Decider
An AI judge system designed to score complex judgments and estimate confidence levels. It supports reporting accuracy within binned confidence intervals and provides calibration curve disclosures.
Core Facts
Timeline (last 90 days)
In the Decider model, developers can set a confidence threshold to automatically hand low-confidence judgments over to human handling.
Decider supports image input, and you need to embed a base64-encoded image_url object into the JSON state field for submission.
Decider has three types of typing questions: Boolean questions return a probability value between 0 and 1; Choice questions return a probability distribution over predefined categories; Score questions return the probability of each score value on an ordered scoring scale.
d1-ultra supports posing multiple questions about the same input within a single request and returning their respective probabilities.
Starting from TypeSafe's release of Jev in mid-September 2026, at least eight companies launched AI decision models within a little over a month, including Cloudflare's Clef, the AWS Strands team's Decider, Perplexity's pplx-decider, Liquid AI's d1 series, OpenAI's Decisions API and Microsoft-Decision-1.
Decision models cannot explain their own judgments, and the Strands team admits that Decider is significantly weaker than reasoning models on complex problems and is not suited to coding, chat or summarization scenarios.
Decision models do not generate text; instead, given an input state and a set of named questions, they return typed answers with probabilities in a single forward pass.
Decider reports a correct rate of 99.6% for judgments at 90% confidence and above on JudgeBench, 94.0% for the 80–90 interval, 84.1% for the 70–80 interval, and 67.7% for below 70%.
A judge that returns a choice, a calibrated confidence level, and a written argument provides users' ledgers with a self-verifiable reference point.
Decider's self-tested consistent accuracy on ContextualJudgeBench under the official paired protocol is 67.1%, compared to the official reference value of 65.4%, with 12 orders excluded due to repeated platform failures.
1 more timeline events
All Facts (11)
Decision models do not generate text; instead, given an input state and a set of named questions, they return typed answers with probabilities in a single forward pass.
65%UnverifiedDecider has three types of typing questions: Boolean questions return a probability value between 0 and 1; Choice questions return a probability distribution over predefined categories; Score questions return the probability of each score value on an ordered scoring scale.
50%UnverifiedDecider supports image input, and you need to embed a base64-encoded image_url object into the JSON state field for submission.
50%UnverifiedIn the Decider model, developers can set a confidence threshold to automatically hand low-confidence judgments over to human handling.
50%Unverifiedd1-ultra supports posing multiple questions about the same input within a single request and returning their respective probabilities.
50%UnverifiedDecision models cannot explain their own judgments, and the Strands team admits that Decider is significantly weaker than reasoning models on complex problems and is not suited to coding, chat or summarization scenarios.
50%UnverifiedOn JudgeBench, Decider made 620 judgments and publicly disclosed 6 first-round failures.
50%UnverifiedDecider reports a correct rate of 99.6% for judgments at 90% confidence and above on JudgeBench, 94.0% for the 80–90 interval, 84.1% for the 70–80 interval, and 67.7% for below 70%.
50%UnverifiedDecider's self-tested consistent accuracy on ContextualJudgeBench under the official paired protocol is 67.1%, compared to the official reference value of 65.4%, with 12 orders excluded due to repeated platform failures.
50%UnverifiedA judge that returns a choice, a calibrated confidence level, and a written argument provides users' ledgers with a self-verifiable reference point.
50%UnverifiedStarting from TypeSafe's release of Jev in mid-September 2026, at least eight companies launched AI decision models within a little over a month, including Cloudflare's Clef, the AWS Strands team's Decider, Perplexity's pplx-decider, Liquid AI's d1 series, OpenAI's Decisions API and Microsoft-Decision-1.
50%