Unverified50% confidenceFactExact time
The v6 dataset was rebuilt around v1's failure modes, including refusal rows, anti-hallucination pairs, calibration examples, and tool trajectories; v2 achieved 9/10 domain tasks, 5/5 honesty, 5/5 refusal capability, and 0/10 hallucination rate on the same evaluation harness, with data composition as the only variable.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedThe study categorized failures into two types: genuine capability deficits and issues that can be mitigated by adjusting the decoding threshold, further subdividing them into 4 failure mechanisms.70% similarUnverifiedAblation testing diagnoses whether a feature is effective by setting its weight to zero one at a time and observing performance changes: if performance is nearly unchanged after disabling it, the feature design is problematic or redundant; a significant drop proves it is contributing.68% similarUnverifiedThe original article's scheme assigns weights to different failure categories: prompt injection failures weight 5, JSON formatting errors weight 3, multi-hop reasoning wrong answers weight 2, with the final score calculated on a weighted basis.68% similarUnverifiedTraditional confidence calibration methods mostly rely on model-generated output text or sequence-based probability statistics, making it difficult to capture complex failure patterns in multi-turn agent interactions.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012407API
curl https://kongchang.com/api/v1/knowledge/claims/1012407MCP
get_claim(id=1012407)