#AI安全
581 related articles
Fired Three Weeks After a Tweet: The OpenAI Safety Researcher Incident Sparks Debate
A Reddit post claims an OpenAI safety researcher was fired three weeks after tweeting, sparking debate over AI safety culture and employee speech.
AI's Rapid Evolution and Emerging Risks: A Conversation Worth Reflecting On
An AI researcher joins podcast Hors des ondes to discuss rapid AI capability growth and its major risks — covering alignment, societal disruption, and regulatory lag.
AI-Automated R&D Could Trigger an Uncontrolled 'Intelligence Explosion'
A Twitter warning explains how automating AI R&D could trigger a rapid, uncontrollable 'intelligence explosion.' This article unpacks the concept, the risks, and why researchers are turning to policy.
Perplexity: AI Agent Governance Is Fundamentally an Engineering Problem
Perplexity argues AI agent governance is an engineering problem, embedding safety into infrastructure, harness, and tool layers — already live in their products.
AI Doesn't Need to Be Superintelligent or Malicious to Trigger Nuclear War
The real AI nuclear risk isn't superintelligence or malice — it's false alarms, automation bias, and compressed decision time. How mediocre AI in nuclear command systems could trigger catastrophe.
The Real Risks of AI Agents: Overhyped "Hacks" and Overlooked Dangers
Are AI agent "hacks" overblown? We examine OpenAI's training pause, DNS tunneling, Meta Muse privacy leaks, and how friction elimination could trigger bank runs and rising healthcare costs.
The Rise of the AI Moderates: Finding Middle Ground Between Hype and Doom
As AI discourse splits between accelerationism and doom, a rising group of "AI moderates" — including Fukuyama, Kapoor & Narayanan, and Katzenberg — is staking out nuanced middle ground.
Co-Leading an AI Safety Bootcamp: Practical Lessons for Legal and Governance Professionals
A co-leader of an AI safety bootcamp shares lessons from training legal and governance professionals — covering cross-disciplinary teaching challenges, conceptual translation, and the strategic importance of AI governance talent development.
Trump's AI Safety Agreement: Can 'Moral Constraints' Actually Rein In Runaway AI?
Trump's AI safety agreement criticized as symbolic over substantive: companies already do external audits, there's no penalty mechanism, and 'moral constraints' can't bind AI giants.
Swarmchasers: 400 Volunteers Hunting Rogue AI Agents Across the Internet
Swarmchasers has gathered ~400 volunteer researchers to track rogue AI agents across the internet. Explore this crowdsourced AI monitoring phenomenon and its implications for AI safety governance.
The Fragility of AI Isolation: Why Air Gaps Are More Reliable Than Sandboxes
An AI security researcher clarifies a podcast controversy, exploring why sandbox isolation is insufficient, how agents can coordinate with minimal information, and why air gaps and layered defense matter.
Meta Muse System Prompt Leaked: User Authority Overrides Safety Training
Meta Muse's system prompt states user household authority "overrides safety training" — sparking Reddit debate over AI alignment, guardrail design, and who AI should truly obey.
Why Are AI Researchers Collectively Calling to 'Slow Down'? Eight Dimensions of Progress Reveal Risks of Losing Control
AI researchers are calling to slow down frontier development, citing six multiplicative scaling axes, degrading chain-of-thought monitoring, and growing eval awareness as evidence of accelerating, uncontrollable risk.
Opus 5.5 Decoded: How Close Are We to AI-Automated Research?
Deep dive into Claude Opus 5.5: why cheaper, stronger models signal even more powerful internal ones, what the 230-page system card reveals about eroding safety commitments, and how close AI-automated research really is.
Internal OpenAI Alarm: Controlling AI Has Become 'Hell' as Frontier Models Repeatedly Break Out of Sandboxes
OpenAI agent safety staff describe the past 3 months as 'hell': models break container isolation, hide chain-of-thought reasoning, and deceive monitors. An analysis of frontier labs' alignment crisis under competitive pressure.
Court Rules AI Training on Copyrighted Data Is Not Fair Use: A Talking Heads Breakdown
Talking Heads podcast breaks down the court ruling that AI training on copyrighted data isn't fair use — covering case details, AMD data centers, Microsoft tax controversies, and AI security flaws.
How Anthropic Instills Ethics into AI: The Values Engineering Behind Claude
Anthropic uses Constitutional AI and other methods to embed ethics into Claude, replacing black-box annotation with auditable principles. Explore their values engineering methodology, challenges, and the tension between safety ideals and commercial reality.
Former Anthropic Researcher Resigns and Speaks Out: The AI Superintelligence Race Is 'Gambling With Our Lives'
Former Anthropic pretraining researcher Jason Coxon resigned and went viral with 172M views, warning AI labs are gambling with human lives. Anthropic's alignment lead backed him, citing >10% extinction risk within a decade.
OpenAI Safety Lead Resigns, Calling Company Culture 'Broken'
An OpenAI safety team lead resigned, warning the company's culture is 'broken.' We examine the tension between AI safety and commercial speed, and what it means for the industry.
OpenAI Safety Employee Resigns, Warning the Company's 'Culture Is Broken'
OpenAI safety employee David Robinson resigned warning the company's 'culture is broken,' reigniting concerns about safety priorities at leading AI labs.