QuALITY
A benchmark dataset for long-document question answering, used to evaluate the ability of models to understand long texts and answer questions about them.
Timeline (last 90 days)
KV-Surgeon was evaluated on three long-context benchmark datasets: QuALITY, LongHealth, and Long Code Arena.
LatComm reduces input by 57 times and speeds up by 3 times on the QuALITY dataset.
QuALITY is a long-document reading comprehension benchmark containing multiple long articles (about 5,000 words on average) and multiple-choice questions.
LatComm speeds up answering all questions about an article by 3 times on the QuALITY dataset.
The paper reports that RAPTOR, paired with GPT-4 on the QuALITY benchmark (long-document question answering), improved absolute accuracy by about 20 points, reaching state of the art at the time.
All Facts (5)
KV-Surgeon was evaluated on three long-context benchmark datasets: QuALITY, LongHealth, and Long Code Arena.
50%UnverifiedLatComm reduces input by 57 times and speeds up by 3 times on the QuALITY dataset.
50%UnverifiedQuALITY is a long-document reading comprehension benchmark containing multiple long articles (about 5,000 words on average) and multiple-choice questions.
50%UnverifiedLatComm speeds up answering all questions about an article by 3 times on the QuALITY dataset.
50%UnverifiedThe paper reports that RAPTOR, paired with GPT-4 on the QuALITY benchmark (long-document question answering), improved absolute accuracy by about 20 points, reaching state of the art at the time.
50%