OpenTWBench
An open evaluation suite focused on Taiwan knowledge and daily life, comprising the academic examination benchmark tw-exam-bench and the contemporary life benchmark tw-formosa-bench. Its emphasis is on quantifying how the evaluation apparatus itself affects model scores.
Timeline (last 90 days)
OpenTWBench is an open evaluation suite built around two benchmarks: tw-exam-bench and tw-formosa-bench.
On the same questions, merely changing the answer extraction method from pattern matching to an explicit \boxed{} format raised accuracy from 20.8% to 93.9%.
Adding just one placeholder character to the system prompt alone can make a certain model jump from last place (58.2%) to first place (83.9%) on the same benchmark.
Doubling the thinking token budget from 8k to 16k brought only a 0.2% accuracy improvement, while changing the distribution of failed questions.
A validation panel composed of three reviewers was found to be essentially 'one reviewer in disguise': two of its members approved 92%–99% of the presented content.
The OpenTWBench team released the entire dataset, per-question difficulty data, the screening list, and a public leaderboard (https://opentwbench.ai).
All Facts (6)
OpenTWBench is an open evaluation suite built around two benchmarks: tw-exam-bench and tw-formosa-bench.
50%UnverifiedOn the same questions, merely changing the answer extraction method from pattern matching to an explicit \boxed{} format raised accuracy from 20.8% to 93.9%.
50%UnverifiedAdding just one placeholder character to the system prompt alone can make a certain model jump from last place (58.2%) to first place (83.9%) on the same benchmark.
50%UnverifiedDoubling the thinking token budget from 8k to 16k brought only a 0.2% accuracy improvement, while changing the distribution of failed questions.
50%UnverifiedA validation panel composed of three reviewers was found to be essentially 'one reviewer in disguise': two of its members approved 92%–99% of the presented content.
50%UnverifiedThe OpenTWBench team released the entire dataset, per-question difficulty data, the screening list, and a public leaderboard (https://opentwbench.ai).
50%