Unverified50% confidenceFactExact time
The paper decomposes the majority-vote supervision error under online direct preference optimization (DPO) into two parts: population-majority bias and prompt-dependent finite-rollout variation.
1
Sources
50%
Confidence
Long-term
Relevance
11/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSystems with the same overall accuracy may produce markedly different error distributions across two population groups, for example frequently wrongly rejecting applicants in group A who would have repaid on time, and more often wrongly approving applicants in group B who ultimately defaulted.65% similarUnverifiedWhen the volume of crowdsourced voting evaluation data is small in the early stages, results may fluctuate and the confidence intervals for rankings are relatively wide.65% similarUnverified"Cheap, Consistent, Calibrated, but Biased?" tested 8 typed decision models (TDMs) with probabilistic outputs and 7 LLM evaluators, finding that each TDM was biased on 3-8 of 12 bias tests, with emotional tone having the largest effect, causing accuracy to plummet from 99% to 42%.64% similarUnverifiedIn a 2023 paper, Anthropic systematically measured sycophancy bias and found that even given obviously incorrect premises, a substantial proportion of responses from most mainstream models still chose to comply with the user.64% similarUnverifiedFitting the propensity score in-sample induces dependence among individual contributions, so variance estimators derived under the i.i.d. assumption underestimate the true uncertainty.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1150281API
curl https://kongchang.com/api/v1/knowledge/claims/1150281MCP
get_claim(id=1150281)