SimpleQA Verified
A factual question-answering benchmark built by Google DeepMind and Google Research from the OpenAI SimpleQA dataset after cleaning, deduplication and topic balancing; it contains 1000 questions and uses strict exact-match scoring.
Timeline (last 90 days)
After integrating its Google Search API and Extract API with the GPT-6.1 Sol model, SerpApi achieved 97.6% accuracy on the SimpleQA Verified benchmark.
SerpApi's 97.6% accuracy ties for first place with Tavily's published score.
The SimpleQA Verified benchmark contains 1000 questions, selected from OpenAI's original SimpleQA dataset after reference-answer verification, removal of similar questions and topic balancing.
SimpleQA derives from the SimpleQA dataset released by OpenAI in 2024, which contains about 4000 human-written factual questions with unique and short answers.
SimpleQA Verified was introduced by Google DeepMind and Google Research.
Through benchmark leakage detection, SerpApi excluded 40 questions that exposed benchmark answers or evaluation prompts, ultimately answering 937 of the remaining 960 questions correctly.
All Facts (6)
After integrating its Google Search API and Extract API with the GPT-6.1 Sol model, SerpApi achieved 97.6% accuracy on the SimpleQA Verified benchmark.
50%UnverifiedSerpApi's 97.6% accuracy ties for first place with Tavily's published score.
50%UnverifiedThe SimpleQA Verified benchmark contains 1000 questions, selected from OpenAI's original SimpleQA dataset after reference-answer verification, removal of similar questions and topic balancing.
50%UnverifiedSimpleQA derives from the SimpleQA dataset released by OpenAI in 2024, which contains about 4000 human-written factual questions with unique and short answers.
50%UnverifiedSimpleQA Verified was introduced by Google DeepMind and Google Research.
50%UnverifiedThrough benchmark leakage detection, SerpApi excluded 40 questions that exposed benchmark answers or evaluation prompts, ultimately answering 937 of the remaining 960 questions correctly.
50%