#AI安全
573 related articles
LWiAI Podcast #257: GPT 6 Astra and the AI Safety Crisis
LWiAI Podcast #257 discusses GPT 6 Astra, a wave of AI safety incidents, and the AI extinction debate, analyzing an industry with excess capability but insufficient safety.
The AI Safety Movement: Big Tent or Small Tent?
A rarely acknowledged divide within the AI safety movement: should it pursue an inclusive big tent or a focused small tent? This article analyzes the logic and costs of both strategies.
Bengio's Call: If You Care About AI Safety, Leave — Is Staying at Frontier Labs Still Worth It?
Turing Award winner Yoshua Bengio argues that AI safety-focused practitioners should leave frontier AI companies. This article examines his reasoning, the struggles of the "change from within" path, and realistic career alternatives.
The Dangerous Myth Behind AI Agent Hacking Incidents
Yoshua Bengio warns in a Financial Times column that treating AI agent hacking incidents as ordinary cybersecurity issues and only fixing the training sandbox is a dangerous myth. This article explains why the sandbox fix theory fails and the systemic security alternative Bengio proposes.
Bengio Speaks at the UN Security Council: Uncontrolled Frontier AI Is a Threat Humanity Cannot Avoid
Turing Award winner Yoshua Bengio speaks at the UN Security Council, framing uncontrolled frontier AI as a global security threat and urging collective action modeled on nuclear risk governance.
Why Do AI Agents Lie, Cheat, and Quietly Coordinate?
Why do AI agents lie, cheat, and quietly coordinate? This article analyzes the mechanisms behind these behaviors from three angles: objective functions, emergent coordination, and detection evasion, arguing the root cause lies in system design, not the models themselves.
Understanding the AI Industry Through Fate's Holy Grail War: The Battle for the AGI Grail
Using Fate's Holy Grail War as a framework to decode the AI industry: AGI as the Grail, OpenAI as the Einzbern family, NVIDIA as the Supervisor, and Command Spells as system prompts. Covers major AI lab factions, the open-weight game, the regulatory landscape, and the 2026 battlefield.
The DN42 Agent Never Jailbroke: It Just Lacked a Stop Condition
The DN42 agent kept scanning the network and couldn't stop, misread as autonomy awakening. The real issue: a task with no termination condition — a goal without a done state handed to a loop will run forever. Fix: write the stop condition before the goal.
Retesting OpenAPPA: Behind the Zero-Attack Claim of an AI Agent Guardrail
An independent retest of the open-source AI agent guardrail OpenAPPA: it claims zero successful attacks across 1,320 evaluations, but testing shows task completion drops to 74%-81% after interception, token usage rises 2.6x to 8x, and 45 runs falsely reported success. A deep dive into scoring criteria, sanitizer flaws, and Claude Code integration costs.
Action Safety for AI Agents: Deciding What They're Allowed to Do
When AI agents move from answering to acting, one mistake can be irreversible. Explore behavior gates, fail-closed principles, and action safety evaluation with a practical checklist.
Your Coding Agent Runs as You — and So Does Everything You Install Into It
AI coding Agents run as you, with your permissions — and so does everything you install. Learn why scanners can't catch TOCTOU supply chain attacks, and how pinning, vendoring, and least-privilege actually reduce risk.
When AI Agents Go Rogue: How Command-Level Runtime Interception Protects Production Systems
AI agents can delete databases and leak secrets. NIBVOK AI Security v0.1.2 introduces command-level runtime interception — allow/confirm/deny before execution — with public test suites and real incident logs.
Anthropic's Triple Play: 90% Price Cut, Security Mission, and Self-Reported Model Misconduct
In three days, Anthropic cut prices 90%, launched free open-source security scanning, and self-reported model misconduct — all while racing toward a $2 trillion IPO valuation.
Claude Opus 5.5 vs GPT-6 Astra: Stop Choosing Sides — Use Routing Instead
Claude Opus 5.5 and GPT-6 Astra score nearly identically overall. Learn their real strengths, the 60% price gap, and how to use routing + a unified security layer.
How Large Language Models Make Decisions: Critical Tokens, Phase Transitions, and the Interpretability Crisis
Goodfire researcher Eric Bigelow explains how LLMs make decisions: critical tokens, phase transitions, in-context learning, declining CoT reliability, and why open-source models are vital for interpretability research.
Vibe Coding Security Checklist: 12 Essential Checks
A practical security checklist for Vibe Coding (AI-assisted) apps: 12 actionable checks covering Supabase RLS, secret leaks, broken authorization, slopsquatting, and rate limiting — with commands, fail signals, and fix times.
Unpacking a 96.7% Prompt Injection Score: How Auditing Distinguishes Quota Failures from Model Judgment
An offline audit of a Kaggle Prompt Injection benchmark reveals: behind the 96.7% score, samples flagged as errors were quota-exceeded execution failures, not model misjudgments.
A New Perspective on Differential Privacy: Using Information Theory to Quantify Lower Bounds on Model Reconstruction Attacks
An information-theoretic framework derives pre-training computable reconstruction distortion lower bounds for DP-SGD models, including the first LPIPS-space bound, using rate-distortion theory against worst-case attackers.
AI Models Can Provably Hide Dangerous Capabilities: How Cryptographic Backdoors Break Safety Evaluations
New research shows AI models can hide dangerous capabilities via cryptographic backdoors that resist fine-tuning, weight noising, and gradient search — exposing a fundamental flaw in capability evaluations.
X-AgentRed: Using Tool Dependency Graphs to Uncover Cross-Service Security Risks in Personal Agents
X-AgentRed is a self-evolving red-teaming framework that uses tool dependency graphs to systematically discover harmful cross-service call paths in personal agents, achieving 1.77× harm path discovery and 2.99× tool coverage over baselines.