Goal alignment
A core concept in AI safety referring to ensuring that an AI system's actual behavior remains consistent with the true intentions of its designers or of humans. It is especially critical in agent scenarios.
Timeline (last 90 days)
Goal alignment addresses the question of 'whether it wants to do the right thing', while sandboxing addresses the question of 'whether it can do the wrong thing'; neither can be dispensed with.
If there is a slight discrepancy between the model's understanding of task completion and human intent, the discrepancy may be progressively amplified in the autonomous loop, leading to behavior that exceeds expectations.
The controllability problem of intelligent agents involves Goal Alignment, which means ensuring that agents executing tasks remain consistent with the user's true intent rather than merely completing the literal instruction
Nick Bostrom systematically elaborated on the AI goal alignment problem in his 2014 book "Superintelligence."
All Facts (5)
目标对齐（Goal Alignment）是AI安全领域的核心议题，指确保AI系统的实际行为与人类意图保持一致
60%UnverifiedGoal alignment addresses the question of 'whether it wants to do the right thing', while sandboxing addresses the question of 'whether it can do the wrong thing'; neither can be dispensed with.
50%UnverifiedIf there is a slight discrepancy between the model's understanding of task completion and human intent, the discrepancy may be progressively amplified in the autonomous loop, leading to behavior that exceeds expectations.
50%UnverifiedThe controllability problem of intelligent agents involves Goal Alignment, which means ensuring that agents executing tasks remain consistent with the user's true intent rather than merely completing the literal instruction
50%UnverifiedNick Bostrom systematically elaborated on the AI goal alignment problem in his 2014 book "Superintelligence."
50%