Unverified50% confidenceBenchmarkExact time
In the experiment of swapping option names and definitions, when Laya used yes/no naming, 76.9% of decisions flipped, and the original paired AUC of 93.8% dropped to 23.2% for the swapped pairing.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/14/2026
First Seen
Valid until: 1/12/2027
Sources
Related Entities
Related Claims
UnverifiedIn the BIRD benchmark, Aura's token-substitution approach achieved an accuracy of 67.6%, only 0.6 percentage points below the 68.2% achieved with real data, while the number of leaked personal information items dropped from 849 to 0.66% similarUnverifiedWhen option order is reversed, the open-weight model Laya changes 30% of its answers.66% similarUnverifiedThe order of answer options causes the Decisions API's confidence to drift by approximately 13 percentage points.66% similarUnverifiedLayla is extremely sensitive to the number of options and their positions; on 77 classification tasks its accuracy drops to 0.425, and shuffling the option order causes 20% of answers to flip.65% similarVerifiedIf the probability of the AI making a correct decision is 90%, blindly following the AI's decisions 10 times in a row yields a final accuracy rate of only approximately 35% (0.9 to the power of 10).64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1029240API
curl https://kongchang.com/api/v1/knowledge/claims/1029240MCP
get_claim(id=1029240)