Regent Chess
A sequential decision-making test environment for evaluating the reasoning capabilities of large language models. It incorporates hidden but trackable state variables, enabling the quantitative comparison of model belief accuracy and supporting reproducibility research.
Timeline (last 90 days)
The study uses a sequential decision-making environment called Regent Chess, in which there is a hidden but variable and precisely recorded state, allowing the model's claimed beliefs to be scored against the true state.
The endpoint metric used in the evaluation is the difference of 'model minus uniform control group', where a positive value represents performing worse than the control.
All Facts (2)
The study uses a sequential decision-making environment called Regent Chess, in which there is a hidden but variable and precisely recorded state, allowing the model's claimed beliefs to be scored against the true state.
50%UnverifiedThe endpoint metric used in the evaluation is the difference of 'model minus uniform control group', where a positive value represents performing worse than the control.
50%