Unverified50% confidenceFactExact time
In a test of 12 questions about a 1,061-line pipeline file (three of which asked about features that do not exist), all three models scored a perfect 12/12 and did not invent any features.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/18/2026
First Seen
Valid until: 11/1/2026
Sources
Related Entities
Related Claims
UnverifiedThe research team conducted zero-shot evaluations of off-the-shelf models on 15 long-context tasks, with test subjects including Gemma-4-31B and Qwen-3.6-27B.69% similarUnverifiedThe limitations of this test include: limited task scale (50 tasks, 3 trials per prompt), style versions written by a single author, length confounding between the florid and normal versions, testing only English and local models below 14B, and the perplexity check covering only three smaller Qwen models and using full-precision weights while actual inference used 4-bit quantization.69% similarUnverifiedAmong all 16 configurations across 4 tasks × 4 models, not a single one passed the qualification threshold.68% similarUnverifiedIn a test of 48 synthesized programming requests, under the default four-option question Laya matched the team's annotation on only 21 of them, with a maximum confidence of only 0.26.68% similarUnverifiedIn an evaluation of 30 answerable questions, off-the-shelf small models under 350MB achieved faithfulness (all assertions can be found in the passages) at most 3 times.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1059537API
curl https://kongchang.com/api/v1/knowledge/claims/1059537MCP
get_claim(id=1059537)