FEVEROUS
A benchmark dataset designed for complex fact verification, requiring systems to process both unstructured text and structured tabular data from Wikipedia simultaneously. Each claim is annotated with a minimal evidence set by human annotators, and scoring applies a dual threshold covering both the answer and the supporting evidence.
Timeline (last 90 days)
The strict score in FEVEROUS is defined as the percentage of claims that have both a correct answer and a complete set of annotated evidence included in the submitted evidence.
FEVEROUS is a complex fact-checking benchmark dataset that requires systems to simultaneously handle unstructured text and structured table data from Wikipedia.
Replacing DCUF evidence with UnifEE evidence improved the strict score by 9.61 percentage points, while the improvement in answer accuracy was only 1.96 percentage points.
The paired 95% confidence interval for the strict score gain from evidence replacement is [8.77, 10.43].
After expanding the context from 256 tokens to 2,048 tokens, the evidence gain under fixed-answer conditions on FEVEROUS improved by 3.84 (Qwen) and 3.10 (Llama) percentage points respectively.
The study proposes a claim-level analysis framework that forms four quadrants by crossing correct/incorrect answers with complete/absent evidence, which can reveal changes masked by aggregate metrics.
All Facts (6)
The strict score in FEVEROUS is defined as the percentage of claims that have both a correct answer and a complete set of annotated evidence included in the submitted evidence.
50%UnverifiedFEVEROUS is a complex fact-checking benchmark dataset that requires systems to simultaneously handle unstructured text and structured table data from Wikipedia.
50%UnverifiedReplacing DCUF evidence with UnifEE evidence improved the strict score by 9.61 percentage points, while the improvement in answer accuracy was only 1.96 percentage points.
50%UnverifiedThe paired 95% confidence interval for the strict score gain from evidence replacement is [8.77, 10.43].
50%UnverifiedAfter expanding the context from 256 tokens to 2,048 tokens, the evidence gain under fixed-answer conditions on FEVEROUS improved by 3.84 (Qwen) and 3.10 (Llama) percentage points respectively.
50%UnverifiedThe study proposes a claim-level analysis framework that forms four quadrants by crossing correct/incorrect answers with complete/absent evidence, which can reveal changes masked by aggregate metrics.
50%