MedMemoryBench
A benchmark specifically designed to evaluate the memory capability of LLMs in longitudinal medical dialogue. It contains dialogue sequences across multiple patients and multiple time points, covering dimensions such as state tracking, person attribution, and safe fact recall, and is used to test whether a memory system can retrieve information at the correct time point for the correct person.
Timeline (last 90 days)
On the MedMemoryBench benchmark, MedGraph with a GPT backbone achieves the highest overall accuracy, leading the strongest baseline by 10.5 percentage points.
On the MedMemoryBench benchmark, MedGraph with a Qwen backbone achieves the highest overall accuracy, leading the strongest baseline by 10.4 percentage points.
MedMemoryBench is a benchmark specifically designed to evaluate LLMs' memory capability in longitudinal medical conversations, and its questions are designed to cover dimensions such as state tracking, person attribution, and recall of safety facts.
All Facts (3)
On the MedMemoryBench benchmark, MedGraph with a GPT backbone achieves the highest overall accuracy, leading the strongest baseline by 10.5 percentage points.
50%UnverifiedOn the MedMemoryBench benchmark, MedGraph with a Qwen backbone achieves the highest overall accuracy, leading the strongest baseline by 10.4 percentage points.
50%UnverifiedMedMemoryBench is a benchmark specifically designed to evaluate LLMs' memory capability in longitudinal medical conversations, and its questions are designed to cover dimensions such as state tracking, person attribution, and recall of safety facts.
50%