Unverified50% confidenceFactExact time
Reasons for the difficulty of alignment include: the more capable a system is, the more likely it may develop secondary goals inconsistent with its training objectives (such as resource acquisition and self-preservation), and its internal decision-making process is difficult for outsiders to scrutinize.
1
Sources
50%
Confidence
Long-term
Relevance
11/1/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedImperceptible progress brings governance difficulties: capability assessment requires cross-domain expertise and is hard to measure uniformly, and the incentive to invest in safety research may be weaker than the incentive to invest in capability improvement.79% similarUnverifiedThe alternative capabilities (problem definition, architectural judgment, etc.) are harder to measure through benchmarks and rely more on domain knowledge and organizational context, so they are unlikely to be standardized or fully automated in the short term.78% similarUnverifiedCapability assessment faces two major difficulties: inconsistent standards (conclusions measured by different organizations vary greatly) and the emergent and context-dependent nature of capabilities (a model that shows no risk today may become dangerous after fine-tuning or combination with other tools).77% similarUnverifiedCapability development has clear technical pathways and commercial incentives, whereas alignment research faces more ambiguous goals and it is very difficult to verify whether a system has truly achieved alignment.76% similarUnverifiedAnthropic said that if tasks more clearly specified scope (including goals, permitted actions, and network boundaries), these failures might have been avoided, but that alignment training itself is not sufficient or fully robust, so it relies on defense in depth.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1123750API
curl https://kongchang.com/api/v1/knowledge/claims/1123750MCP
get_claim(id=1123750)