Forced Choice
A measurement method commonly used in psychology and political science research that requires subjects to choose between two options, revealing latent preferences and preventing neutral or uncertain responses from masking true tendencies. In AI evaluation it is used to expose directional bias in a model's underlying probability distribution.
Timeline (last 90 days)
Forced-choice is a measurement method commonly used in psychology and political science research, revealing subjects' underlying preferences by requiring them to pick one of two options.
In AI evaluation, forced binary choice can expose leanings that models may hide during free generation; however, this method measures the model's output behavior under specific prompts rather than the model's "internal beliefs," and the results are highly sensitive to prompt wording, option order and context.
All Facts (2)
Forced-choice is a measurement method commonly used in psychology and political science research, revealing subjects' underlying preferences by requiring them to pick one of two options.
50%UnverifiedIn AI evaluation, forced binary choice can expose leanings that models may hide during free generation; however, this method measures the model's output behavior under specific prompts rather than the model's "internal beliefs," and the results are highly sensitive to prompt wording, option order and context.
50%