Unverified50% confidenceFactExact time
The order of answer options causes the Decisions API's confidence to drift by approximately 13 percentage points.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedLayla is extremely sensitive to the number of options and their positions; on 77 classification tasks its accuracy drops to 0.425, and shuffling the option order causes 20% of answers to flip.68% similarUnverifiedRanked by question difficulty, persistence rate was 91.4% on the 60 questions that all models answered correctly, but only 73.5% on the 52 questions that at least one model got wrong, indicating that sycophancy tracks uncertainty.66% similarUnverifiedIn 11 of the 12 models, between 87.5% and 100% of capitulation cases stopped at the exact wrong number provided by the user, meaning capitulation typically involves switching sides entirely.65% similarUnverifiedWhen option order is reversed, the open-weight model Laya changes 30% of its answers.65% similarUnverifiedDirectly asking a large model to score candidates often results in scores concentrated in a certain range (e.g., 7–9 points), lacking differentiation.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1002802API
curl https://kongchang.com/api/v1/knowledge/claims/1002802MCP
get_claim(id=1002802)