ALIGNMENTFORECASTBENCH
A benchmark for measuring progress on alignment forecasting tasks. It covers over 5,000 forecasting questions, 17 target models, 32 datasets, and 16 failure modes, designed to evaluate the generalization of forecasting methods across different models, data sources, and failure types.
Timeline (last 90 days)
The researchers constructed a benchmark test set called ALIGNMENTFORECASTBENCH, covering more than 5,000 prediction questions, 17 target models, 32 datasets, and 16 failure modes.
Frontier large models perform poorly at directly predicting alignment risks on ALIGNMENTFORECASTBENCH.
All Facts (2)
The researchers constructed a benchmark test set called ALIGNMENTFORECASTBENCH, covering more than 5,000 prediction questions, 17 target models, 32 datasets, and 16 failure modes.
50%UnverifiedFrontier large models perform poorly at directly predicting alignment risks on ALIGNMENTFORECASTBENCH.
50%