SRBench
A benchmark framework designed specifically for systematic review literature screening tasks. It contains 45,064 annotated entries spanning 32 secondary studies and employs evaluation metrics tailored to class-imbalance problems, enabling assessment of LLM performance on real-world literature screening tasks.
Timeline (last 90 days)
A paper proposes SRBench, a benchmark framework targeting the screening phase of systematic reviews, along with a companion experimental tool PromptSR for evaluating large language models on literature screening tasks.
The SRBench benchmark dataset contains 45,064 annotated entries.
The SRBench dataset covers 32 carefully curated secondary research studies.
SRBench's evaluation framework is specifically designed for class imbalance problems, explicitly accounting for the high proportion of excluded articles relative to included articles in systematic reviews.
The paper presents an application case demonstrating how SRBench and PromptSR work together.
All Facts (5)
The SRBench benchmark dataset contains 45,064 annotated entries.
50%UnverifiedThe SRBench dataset covers 32 carefully curated secondary research studies.
50%UnverifiedSRBench's evaluation framework is specifically designed for class imbalance problems, explicitly accounting for the high proportion of excluded articles relative to included articles in systematic reviews.
50%UnverifiedThe paper presents an application case demonstrating how SRBench and PromptSR work together.
50%UnverifiedA paper proposes SRBench, a benchmark framework targeting the screening phase of systematic reviews, along with a companion experimental tool PromptSR for evaluating large language models on literature screening tasks.
50%