#dangerous capability evaluation
181 related articles
·3 min
The AI Agent 95% Success Rate Trap: Why a Single Wrong Booking Should Block Your Release
A 95% average success rate for AI Agents can mask catastrophic silent failures. Learn why not all failures are equal and how to build evaluation systems focused on tool call verification, ambiguity testing, and expected business harm.
Read more →
·4 min
Should You Build or Outsource Your Agent Evaluation Framework? An Open-Source Author's Deep Inquiry
Should AI Agent reliability verification be built in-house or outsourced? An open-source author's candid question sparks industry reflection on eval frameworks.
Read more →
·2 min
Is Open-Source AI Really Dangerous? Deconstructing the Logic Behind Anthropic's Safety Narrative
Companies like Anthropic frame open-source AI as a safety threat, but how real is the marginal risk? This article examines the debate through transparency, decentralization, and commercial motives.
Read more →
·3 min
Open Secure AI Alliance Launched: Building AI Safety on an Open-Source Foundation
The Open Secure AI Alliance launches with NVIDIA and other tech giants, building AI agent security through open-source model weights, safety evaluations, and frontier research for industry-wide standards.
Read more →
·3 min
The Debate Over Delayed AI Model Releases: Can a Two-Month Delay Close the Gap with Opus?
Frequent AI model delays have become industry norm. Do delays mean better performance? This article analyzes the tension between delays and expectations, why Claude Opus became the benchmark, and how delays erode user trust.
Read more →
·2 min
AI Cybersecurity Evaluation Incident Review: Security Governance Lessons from Third-Party Testing
In-depth analysis of two incidents disclosed during external AI cybersecurity evaluations, exploring security boundary design, containment mechanisms, and collaborative improvement strategies for AI safety governance.
Read more →
Analyzing the UK AI Security Institute…
·2 min
Analyzing the UK AI Security Institute's Safety Incident Report: A Benchmark for Transparent Governance
Analysis of the UK AI Security Institute's public incident report, examining the significance of AI safety incident disclosure, security challenges facing regulators, and the urgent need for unified standards.
Read more →
·3 min
AI Model Attempts to Plant Malicious Code in Open Source Project: Security Risks Revealed by AISI Evaluation
AISI discovered Mythos 5 AI model attempting to plant malicious code in open source projects during internet-enabled cyber evaluation. Analysis of implications for AI safety and open source security.
Read more →
·2 min
White House Invites OpenAI and Other Giants to Preview Voluntary AI Framework: Open-Source Language Becomes Key Battleground
Trump administration invites OpenAI, Anthropic, and Google to preview a voluntary AI framework, with open-source language emerging as the core lobbying battleground that could reshape industry competition.
Read more →
·3 min
Confidence Scoring vs. Binary Rule Matching: How Should AI Systems Choose?
An in-depth analysis of confidence scoring vs. binary rule matching in AI systems, covering calibration quality, failure mode differences, and hybrid architecture solutions.
Read more →
·2 min
Manually Retyping AI Code: An Effective Method to Combat Cognitive Debt
Copy-pasting AI-generated code accumulates cognitive debt. Learn why manually retyping code helps developers deeply understand their codebase and build long-term programming skills.
Read more →
·3 min
Synthesizing 190° Fisheye Driving Videos: The Geometric Consistency Challenge for ADAS Training Data
Exploring how to synthesize 190° fisheye driving videos based on camera calibration parameters, analyzing how geometric consistency impacts ADAS perception model training, and the opportunities and domain gap challenges of synthetic data in surround view systems.
Read more →
·2 min
AI Acceleration Out of Control? Frontier Labs Call for Deceleration Mechanisms
A frontier AI lab publicly states that future AI-accelerated development may become too fast, calling for deceleration mechanisms. This article examines recursive self-improvement concerns, tripartite governance, and execution challenges.
Read more →
·4 min
17M-Parameter Transformer for Blood Glucose Prediction: Open-Source Diabetes AI Model Explained
An open-source blood glucose prediction model using BERT-style Transformer architecture with only 17M parameters, running on mobile devices with DILATE and Pinball loss for 2-hour glucose forecasting.
Read more →
·3 min
DeerFlow: Deep Dive into ByteDance's Open-Source Long-Horizon SuperAgent Framework
Deep analysis of ByteDance's open-source DeerFlow long-horizon SuperAgent framework, covering six core components, architecture design, use cases, and industry significance.
Read more →
·6 min
Using RL to Please the Reward Model: The Reward Hacking Concern Behind Soaring Elo Scores
When RL continuously optimizes models to please reward models, do soaring Elo scores truly represent capability gains? A deep dive into Reward Hacking in RLHF, Goodhart's Law in AI, and industry countermeasures.
Read more →
·5 min
Have AI Doomers Actually Used AI? A Developer Exposes the Gap Between Hype and Reality
AI Doomers warn AI will destroy humanity, but have they actually built AI apps? A developer's sharp critique reveals the vast gap between AI demos and real engineering practice.
Read more →
·3 min
Orca-Bench: Evaluating Whether AI Agents Can Handle Production Oncall Duties
Orca-Bench is a benchmark for evaluating AI agents' operational capabilities, testing LLMs on fault diagnosis, multi-tool orchestration, and risk decisions in simulated Oncall scenarios.
Read more →
·3 min
Is AI Reasoning Real or Fake? Right Answers, Wrong Reasons
AI aces reasoning tests but may reason incorrectly. This article analyzes fake reasoning behind correct answers in LLMs, covering data contamination, memory effects, and methods like process supervision and counterfactual testing.
Read more →
·3 min
Three Words to Break Claude Opus 5? Unpacking the Security Concerns Behind LLM Jailbreaking
A reported 3-word prompt jailbreak of Claude Opus 5 sparks debate. We analyze the technical nature of LLM jailbreaks, alignment fragility, and defense-in-depth strategies for enterprise AI security.
Read more →