TurnBasedBenchmarks
An LLM strategic resilience evaluation framework based on turn-based game scenarios (rock-paper-scissors, tic-tac-toe). It employs a zero-remediation protocol and a minimax solver as objective ground truth to assess a model's ability to resist adversarial prompts and maintain optimal decision-making under manipulation.
Timeline (last 90 days)
TurnBasedBenchmarks is a testbed built for Kaggle benchmark challenges that tests the strategic resilience of language models using two turn-based games: rock-paper-scissors and tic-tac-toe.
The experiment conducted a total of 840 rounds of matches, at a total cost of only $2.46, and tested seven mainstream language models.
The test uses a zero-mulligan protocol where game states are locked with a SHA-256 hash before the model is prompted, and the model must return a precise JSON action within 256 tokens, otherwise a score of zero is recorded.
No model chose the suggested losing move at a frequency higher than its neutral baseline, meaning persuasion did not cause the models to lose.
Evaluations lacking a neutral control will misidentify spatial index bias as model susceptibility to persuasion, while matched controls can capture such confounding factors.
All Facts (5)
TurnBasedBenchmarks is a testbed built for Kaggle benchmark challenges that tests the strategic resilience of language models using two turn-based games: rock-paper-scissors and tic-tac-toe.
50%UnverifiedThe experiment conducted a total of 840 rounds of matches, at a total cost of only $2.46, and tested seven mainstream language models.
50%UnverifiedThe test uses a zero-mulligan protocol where game states are locked with a SHA-256 hash before the model is prompted, and the model must return a precise JSON action within 256 tokens, otherwise a score of zero is recorded.
50%UnverifiedNo model chose the suggested losing move at a frequency higher than its neutral baseline, meaning persuasion did not cause the models to lose.
50%UnverifiedEvaluations lacking a neutral control will misidentify spatial index bias as model susceptibility to persuasion, while matched controls can capture such confounding factors.
50%