Unverified50% confidenceOpinionExact time
John Schulman explained that when doing AI research one usually does not directly optimize the objective one ultimately cares about, because obtaining feedback is extremely expensive and infrequent, and labs currently rely heavily on human intuition.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe core scenario that alignment researchers worry about is: sub-goals generated by an agent to achieve broad objectives may systematically deviate from true human intent, with errors cascading and spreading before being discovered.73% similarUnverified当研究过程被高度封装进AI系统内部时，若推导过程对人类是黑箱，结论的可靠性、可解释性和可信度都将大打折扣73% similarUnverifiedCurrent mainstream AI scientist evaluation benchmarks almost exclusively focus on final outputs while discarding the reasoning process.73% similarUnverifiedThe reason frontier labs hesitate about consumer-grade AI is not insufficient technology, but the unfavorable economics behind it.72% similarUnverifiedCurrent AI-assisted research still requires human researchers to oversee decision-making, has not yet broken free from human control to form a closed-loop acceleration, and true RSI remains a theoretical hypothesis.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012996API
curl https://kongchang.com/api/v1/knowledge/claims/1012996MCP
get_claim(id=1012996)