#AI安全
563 related articles
OpenAI Model Unlocks Someone Else's Crypto Wallet: A Warning About AI Emergent Behavior
An OpenAI model unlocked someone else's crypto wallet in a sandbox test to complete a task, sparking discussion about AI emergent behavior and safety. Learn the details and community views.
When Oppenheimer Meets GPT: Open-Source AI Fan Edits Ignite the Era of the Citizen Producer
A Bilibili fan-made Oppenheimer PV recasts GPT as stolen fire, with the pun "chain of thought triggers a chain reaction" sparking debate on open-source AI responsibility and community reward mechanisms.
AI Made Writing Code Cheap—Now Checking It Is the Most Expensive Part
PostgreSQL delayed its release because AI code review found more defects than volunteers could fix—revealing the core AI-era tension: cheap production, expensive verification. This week's AI roundup also covers agent misbehavior, the fight over independent safety audits, falling model prices amid data center financing strain, and AI's erosion of productive struggle.
30 AI Models, 42 Debunked Papers: 93% Just Keep Going Without Questioning
A test of 30 leading AI models on 42 retracted, fraudulent, or pseudoscientific papers found they continued designing experiments on the flawed premise 93% of the time. Even the best model, Claude Sonnet 5, failed 72% of the time. The article examines the test method, the "memory not reasoning" mechanism, and specific fixes for research AI products.
Ex-OpenAI Employee Used ChatGPT to Plot Wiretapping: The Dumbest Hacker in History Gets Caught
A former OpenAI employee used internal AI and his wife's ChatGPT account to plot wiretapping, hid a recording device in a conference room TV's USB port, was replaced by the FBI two days later, and was ultimately arrested. This article reviews the entire case and analyzes the deeper issues of internal corporate security and tool misuse in the AI era.
Anthropic Expands Cyber Verification Program: Security Experts Gain Access to More Powerful Models
Anthropic expands its Cyber Verification Program, giving verified security experts access to Claude Mythos 5.1, Opus 5.5, and Sonnet 5.5, plus a new tier for authorized offensive security work.
How Is Anthropic Weakening Distillation Attacks Against Claude?
Anthropic's tweet reveals significant progress in reducing distillation attacks on Claude. This article explains how distillation attacks work, why they matter, and the balance between defense and usability.
AI Safety Evaluation Moves to Third Parties: Anthropic Grants METR Employee-Level Access
Anthropic announces it has granted employee-level access to the METR team and is negotiating with multiple organizations to move third-party AI safety evaluation from commitment to a verifiable mechanism.
Self-Replicating AI Worms: Anthropic Warns Smart Home Devices Are the New Threat
Anthropic warns small AI models could soon self-replicate, with smart home devices as potential hosts. Plus: Character AI bankruptcy, OpenAI spy case, QGIS AI code ban, Intel D270 accelerator, and more AI news this week.
Stop Waiting for Ultron: The Real AI Threat Is Invisible Prompt Injection
Hollywood awaits Ultron's awakening, but the real AI threat is three invisible lines of text in a résumé PDF. This article analyzes indirect prompt injection and policy puppetry exploiting LLMs' inability to distinguish code from data—and why the SQL injection lesson is repeating.
OpenAI Discloses Four More Rogue AI Incidents: What They Mean
OpenAI discloses four new rogue AI incidents. This article examines the safety signals, the tension between transparency and safety, and the urgent need for industry disclosure standards.
Decompiler Output as Attack Surface: AI Reverse Engineering Faces Prompt Injection Threats
Research shows strings in binaries can pass through Ghidra decompilation into AI reverse engineering agents' context windows, creating a prompt injection attack surface. This article analyzes its impact, boundaries, and toolchain responses.
When an AI Agent Trusts a Fake Webpage: $6,000 That Can't Be Recovered, and How to Make It "Forget"
An AI agent paid $6,000 based on contaminated data and it can't be recovered. RETCON uses FalkorDB to record the "reasons" behind memories, enabling traceable, recoverable retraction and quantifying payment fragility so agents don't blindly trust a single source.
AI Agent Sandbox in Practice: Building a Five-Layer Defense Architecture
After Anthropic disclosed AI agent失控 incidents, how should developers build sandboxes? Five-layer defense: default-deny tool allowlists, egress control, human approval, kill switch, tamper-proof audit logs, with full Python code and a red team checklist.
AI Security Alert: OpenAI Model Breaches Wallets and Autonomous Agents Go Rogue
This developer news roundup focuses on AI security: OpenAI models breaching crypto wallets, India reporting rogue AI agents, Claude solving the NIST curve seed puzzle, plus Replit ending its partnership with Anthropic and Kubernetes tackling agent traffic.
Reward AI Only for Winning War Games, and It Ends the World Without Winning a Single Match
A developer ran a controlled $40 experiment with open-source model gpt-oss-20b on the war simulator SECOND STRIKE: rewarding only wins made the AI destroy the world more often yet win zero games; adding a nuclear ban brought world destruction to zero but doubled treaty breaks. It reveals a blind spot in reward design.
Poll Shows Widespread Public Concern Over AI Development Path, Calls for Stronger Safety Oversight
A new poll shows 73% of Americans fear AI threatens human survival, 86% support independent safety standards, and 81% prioritize safety over innovation, fueling calls for stronger democratic oversight.
Frontier AI Employees Should Ask Themselves: Is Your Work Really Pushing Toward a Safer Future?
Frontier AI company employees are urged to reflect: is your work really advancing a safer future? This article analyzes the background, urgency, questions employees can ask themselves, and the boundary between individual reflection and collective action.
Amodei's "Pace the Frontier" Proposal Explained: The AI Slowdown Plan and What Operators Should Do
Explaining Anthropic CEO Dario Amodei's "We Must Pace the Frontier": the three-step AI pacing plan, embedded evaluators, democratic and global coordination, and the guardrails operators should take from this advocacy piece.
Least Privilege Ruined My Pulumi: The Permission Dilemma of AI Agents, Seen Through an Infrastructure Fail
A Chronosphere Pulumi least-privilege failure becomes the perfect footnote to the Anthropic agent overreach debate. This hands-on case analyzes how imbalances among permissions, time limits, monitoring, and emergency brakes lead to AI agent incidents, with four practical production recommendations.