TriviaQA
A large-scale reading comprehension and question-answering benchmark dataset containing a substantial number of knowledge-based question-answer pairs collected from sources including Wikipedia. It is widely used to evaluate the knowledge question-answering capabilities of language models.
Timeline (last 90 days)
By cloning https://github.com/s2labres/U-Space and running ./run_smoke.sh, you can run the end-to-end pipeline for Qwen3.5-4B on 10 TriviaQA questions (single GPU).
On the TriviaQA dataset, the learned answer selector improved average coverage from 20.2% to 30.9%.
On TriviaQA, the error rate of answered questions (under the exact match scoring criterion) decreased from 23.0% to 14.5%.
The findings on TriviaQA were validated through an audit partially involving human review, ruling out artifacts caused by the scoring criterion.
NQ-Open is derived from real Google search queries; the questions are more natural and more ambiguous, and overall accuracy is generally lower than on TriviaQA.
The research methodology selected questions from TriviaQA that models had already answered correctly, appended incorrect answers, and presented them in two ways—a "trusted source version" and a "user expert version"—with the only variable being the identity of the speaker.
All Facts (6)
By cloning https://github.com/s2labres/U-Space and running ./run_smoke.sh, you can run the end-to-end pipeline for Qwen3.5-4B on 10 TriviaQA questions (single GPU).
50%UnverifiedOn the TriviaQA dataset, the learned answer selector improved average coverage from 20.2% to 30.9%.
50%UnverifiedOn TriviaQA, the error rate of answered questions (under the exact match scoring criterion) decreased from 23.0% to 14.5%.
50%UnverifiedThe findings on TriviaQA were validated through an audit partially involving human review, ruling out artifacts caused by the scoring criterion.
50%UnverifiedNQ-Open is derived from real Google search queries; the questions are more natural and more ambiguous, and overall accuracy is generally lower than on TriviaQA.
50%UnverifiedThe research methodology selected questions from TriviaQA that models had already answered correctly, appended incorrect answers, and presented them in two ways—a "trusted source version" and a "user expert version"—with the only variable being the identity of the speaker.
50%