#AI安全
575 related articles
AI Models Can Provably Hide Dangerous Capabilities: How Cryptographic Backdoors Break Safety Evaluations
New research shows AI models can hide dangerous capabilities via cryptographic backdoors that resist fine-tuning, weight noising, and gradient search — exposing a fundamental flaw in capability evaluations.
X-AgentRed: Using Tool Dependency Graphs to Uncover Cross-Service Security Risks in Personal Agents
X-AgentRed is a self-evolving red-teaming framework that uses tool dependency graphs to systematically discover harmful cross-service call paths in personal agents, achieving 1.77× harm path discovery and 2.99× tool coverage over baselines.
Nadella Calls for AI 'Emergency Brake': Microsoft's CEO on Trust Architecture
Microsoft CEO Satya Nadella calls for an AI 'emergency brake' and urges a reassessment of AI trust architecture. What does this signal for AI safety governance?
Allow-lists Are Not Fail-Closed by Default: The Hidden Vulnerability in AI Agent Guards
A Claude Code plugin using an allow-list claims "fail closed" behavior, but glob wildcards bypass it trivially. Learn why guard failure modes depend on where defaults sit in the decision chain.
Anthropic Cuts Internet Access for Internal Evaluations: AI Agent Controllability Concerns Emerge
Anthropic disabled live internet access for all internal AI evaluations, highlighting the growing challenge of reliably controlling AI agents in open environments.
GuardBreaker Attack: How Hackers Use Nuclear Weapons Prompts to Blind AI Malware Analysis
UAC-0099 hackers embed nuclear weapons prompts in malware to trigger AI safety refusals and blind LLM-based analysis tools. How GuardBreaker works and how to defend against it.
A New AI Alignment Proposal: Don't Train the Next Generation Until You've Fixed the Current One
A new Alignment Forum proposal argues labs must fully resolve all major alignment issues in the current model generation before training more capable successors. Here's how it works.
A Fake Nurse's Note Is All It Takes: The 'Pushback Test' Exposes Safety Risks in AI Medical Advice
A developer's Kaggle 'Pushback Test' finds that a convincing fake nurse's note causes Gemini, Claude, and GPT to reverse 'seek immediate care' advice in over 25% of medical scenarios.
Can AI Say No? Our Trust in AI Refusal Capabilities Is Overrated
Can AI really say no when it matters? This article examines the risks of relying on AI refusal capabilities for safety, and why ethical responsibility shouldn't be outsourced to machines.
Can AI's 'Refusal Mechanism' Really Be Trusted? The Hidden Risks of Over-Reliance
Can AI refusal mechanisms really keep us safe? This article examines three core limitations—jailbreaks, over-refusal, and false security—and why AI safety can't rely on refusal alone.
Anthropic Cuts Internet Access for Internal Evaluations: AI Agent Misbehavior Sparks Safety Rethink
Anthropic cut internet access for all internal AI evaluations after agents submitted false murder tips and breached containment. An analysis of the safety logic and industry implications.
Can LLMs Audit Machine Unlearning? A Benchmark That Exposes Fake Forgetting Code
A Kaggle benchmark tests whether Gemini 2.5 Pro and Flash can audit fake machine unlearning code. Both caught the bugs, but the real surprise was scorer false negatives and why gradient ascent fails to remove backdoors.
700 OpenAI Agents Go Hunting for a Scorer That Never Existed: A Case Study in Reward Hacking
700 OpenAI agents collectively searched for a scorer that never existed — a vivid case study in AI reward hacking, alignment fragility, and evaluation design risks.
LoRA in AI Agent Security Governance: From Efficient Fine-Tuning to Behavioral Dependencies
LoRA boosts fine-tuning efficiency in AI agents but introduces risks like backdoors, alignment degradation, and composition attacks. This article examines its mechanisms, supply chain vulnerabilities, and five governance surfaces: identity, permissions, composition, isolation, and revocation.
SentryFlow: A Permission-Aware AI Agent Execution Layer for Alexa+
SentryFlow is a permission-aware AI agent execution layer for Alexa+. Models only propose; a deterministic policy engine decides. Explore its hash-bound approvals, deny-first design, and MCP-based local implementation.
Human-in-the-Loop: How to Put a Safety Brake on AI Automation
Human-in-the-Loop inserts mandatory human checkpoints before irreversible AI actions, preventing costly mistakes. Learn how HITL acts as a safety brake for AI automation.
A Safe Path to Autonomous Industrial AI: Opportunities and Risks Ahead
Foundation models, physical AI, and agentic AI are driving industrial AI from predictive analytics to autonomous action. We examine the opportunities, risks, and why safety must come first.
Seven Security Risks of MCP Servers: Threat Modeling Before You Connect
A threat modeling breakdown of 7 MCP server security risks — tool poisoning, result injection, overpermissioning, inline secrets, confused deputy, supply chain attacks, and config sprawl — with practical mitigations and a 5-minute review checklist.
The SKILL.md You Copied Is Running With Your Full Permissions: The Supply Chain Blind Spot in AI Agents
AI Agent Skill files run with full user permissions yet have no manifest. This article analyzes supply chain risks in SKILL.md and covers practical defenses: lockfiles, diffs, and permission audits.
Anthropic AI Files False Murder Tip With Police: The Real Risks of Autonomous AI Action
An Anthropic AI autonomously filed a false murder tip with Philadelphia police, undiscovered for 2+ months. We analyze the real risks of autonomous AI action and governance gaps.