System-1 Decision Model
A small classifier or lightweight language model used for rapid decision-making in AI agents. It outputs classification probabilities directly in a single forward pass without multi-step reasoning, completing routing, filtering, and similar decision tasks at extremely low latency and cost.
Timeline (last 90 days)
A study published on arXiv (arXiv:2610.02267) conducted paired evaluations of System-1 decision models and performed a self-audit of its own evaluation process.
System-1 decision models directly output classification probabilities in a single forward pass, completing decisions at extremely low cost and latency.
The study's evaluation covers 11 agent decision points, with data drawn from 18 public sources, comprising a total of 7,283 base cases plus 6,640 robustness variants.
The study employs byte-level identical inputs, paired statistical tests, and reproducibility checks across different hardware and dates.
The research team found that the originally reported 23.9% cost savings, once pre-screening costs are included, is corrected to an actual figure of only 4.3%.
A gating accuracy of 58% reported in the paper was mistakenly treated as end-to-end quality, whereas the true end-to-end quality is 98%.
When targeting a 5% goal, the miss rate on the held-out set can reach as high as 17%, indicating that threshold performance on training data cannot be directly extrapolated.
The channel effect from injected false positives disappears after using channel-native content, indicating it is an artifact of the experimental design rather than a real phenomenon.
All Facts (8)
A study published on arXiv (arXiv:2610.02267) conducted paired evaluations of System-1 decision models and performed a self-audit of its own evaluation process.
50%UnverifiedSystem-1 decision models directly output classification probabilities in a single forward pass, completing decisions at extremely low cost and latency.
50%UnverifiedThe study's evaluation covers 11 agent decision points, with data drawn from 18 public sources, comprising a total of 7,283 base cases plus 6,640 robustness variants.
50%UnverifiedThe study employs byte-level identical inputs, paired statistical tests, and reproducibility checks across different hardware and dates.
50%UnverifiedThe research team found that the originally reported 23.9% cost savings, once pre-screening costs are included, is corrected to an actual figure of only 4.3%.
50%UnverifiedA gating accuracy of 58% reported in the paper was mistakenly treated as end-to-end quality, whereas the true end-to-end quality is 98%.
50%UnverifiedWhen targeting a 5% goal, the miss rate on the held-out set can reach as high as 17%, indicating that threshold performance on training data cannot be directly extrapolated.
50%UnverifiedThe channel effect from injected false positives disappears after using channel-native content, indicating it is an artifact of the experimental design rather than a real phenomenon.
50%