Unverified50% confidenceFactExact time
On 160 OWASP cases, different model configurations trade off between "cleared" and "unresolved" outcomes; the ten-context experimental group evaluated only 96 cases, exposing limitations in evaluation coverage.
1
Sources
50%
Confidence
Long-term
Relevance
10/28/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn tests with two sets of 120 evidence packages each, the model rejected 70% to 100% of the applicable counterexamples, while checks constructed against specific scopes correctly classified all evidence packages.71% similarUnverifiedThe coverage gate benchmark run found only 8 of the 20 cases, failing to meet the pass condition.71% similarUnverifiedA first round of 80 cases is a reasonable lower bound for paired testing; the power of paired testing comes from discordant pairs, and a set where everything passes is useless no matter how large it is.70% similarUnverifiedIn few-shot scenarios (e.g., approximately 20 samples per class), multi-class subdivision should be abandoned, all defects should be collapsed into a binary classification problem (defective/non-defective), and specific classification should be left to manual post-processing.69% similarUnverifiedThe experiments include: a successful run (all 4 test cases passed), a controlled failure experiment (modifying the discount calculation logic caused failures, and the model pinpointed the problem to the percentage discount formula), and historical analysis (comparing 15 historical reports to verify the transition from failure to pass).69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1106289API
curl https://kongchang.com/api/v1/knowledge/claims/1106289MCP
get_claim(id=1106289)