Unverified50% confidenceBenchmarkExact time
When the authors' answers were explicitly listed, recipients could reproduce their preference configurations with fidelity of 0.955 to 0.960 across the three model grids, within only 0.03 of the fidelity of repeated testing of the same model.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOn the author's internal document corpus, the median Top-1 score for questions with true answers using the ada-002 model is approximately 0.84, while between random unrelated chunks it is approximately 0.72.73% similarUnverifiedAmong the five evaluators the authors tested, if the answer key had minor errors, the score was only 0.21 — lower than giving no key at all (0.46) and lower than giving a completely irrelevant key (0.50); while a human-verified model-generated answer scored 0.61, close to the 0.69 of a human-written one.72% similarUnverifiedIn the selective classification framework, when model confidence is below the threshold τ=0.934, the sample is rejected and referred to an expert for review.71% similarUnverifiedNative self-descriptions (preference configurations measured under a general assistant prompt) showed no reconstruction gain on average; under every tested system-prompt condition, the one-sided 95% upper confidence bound on the pooled gain was below +0.01.71% similarUnverifiedWhen testing prompt injection, change only one variable at a time, and since model behavior is probabilistic, tests should be repeated multiple times and results reported as a rate (for example, three successes out of ten).71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1114297API
curl https://kongchang.com/api/v1/knowledge/claims/1114297MCP
get_claim(id=1114297)