Unverified50% confidenceBenchmarkExact time
After moving the failure-logging instruction from a footnote into a numbered task step, the weak model's feedback-logging score rose from 0.00 to 1.80 (out of 2), and across twenty runs neither model triggered the instruction in the footnote version.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn the experiments, the task resolution rate was 17% for zero-shot generation, 27% for generic self-correction, 23% for error-only feedback, and 90% for diagnostic counterexample feedback (A-CEGIS).75% similarUnverifiedPrompt Guard 2 scored 0.999 on human-written injection instructions, but scored between 0.004 and 0.140 on real AgentDojo attacks, all below the default threshold of 0.5.69% similarUnverifiedThe benchmark produces two numbers for each model that are never merged: a task score and a false-confidence rate (the proportion of cases that should have been escalated but were answered anyway).69% similarUnverifiedThe same scaffold raised the weak model's execution quality from 0.88 to 2.0 (across four scoring dimensions), yet pushed the strong model toward overcautious shutdown (all four dimensions dropped).69% similarUnverifiedModels' standard accuracy on the passage-level ranking task was only 0% to 13%.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1027069API
curl https://kongchang.com/api/v1/knowledge/claims/1027069MCP
get_claim(id=1027069)