Unverified50% confidenceBenchmarkExact time
在漏洞检测基准测试中，让最新模型运行五次，只有50%的漏洞被稳定发现，相比确定性检查工具LLM只发现75%的问题，F1分数仅为40%
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/14/2026
First Seen
Valid until: 11/12/2026
Sources
Related Claims
UnverifiedIn vertical domains such as industrial defect detection and pathology slide analysis, the zero-shot detection accuracy of large models such as GPT-4V and Gemini lags behind specialized models by 20–40 percentage points on the mAP metric.66% similarUnverified在5%显著性水平下测试100个随机因子，期望会有5个看起来显著（多重假设检验问题）63% similarUnverified该测试样本规模相对有限（5个模型、13个代码库），结论有待更大规模验证63% similarUnverified在FastAPI开源项目（53个文件）上用20个问题基准测试，不使用工具每问83,000 tokens，使用本地索引工具每问4,900 tokens，减少94%62% similarUnverified2024年一项研究对GPT-4在Temperature=0下的代码生成任务进行100次测试，发现约8%的测试用例存在跨次运行的输出差异62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/746378API
curl https://kongchang.com/api/v1/knowledge/claims/746378MCP
get_claim(id=746378)