LExMem
A graph-guided benchmark for evaluating memory capability in long-horizon exploration tasks. Its question-answering instances are derived from graph structures and require traversing across branches and tracing revision histories in order to be answered.
Timeline (last 90 days)
Research from the MBZUAI NLP team proposes the graph-guided benchmark LExMem and the graph-based memory framework LExGraph for memory modeling and evaluation in long-horizon exploration tasks.
The LExMem benchmark contains 224 dialogues and 2,018 question-answering instances, all derived from graph structures.
The question-answer pairs in the LExMem benchmark are not manually written and then directly paired with context, but automatically derived from graph structures to ensure that questions depend on cross-branch and cross-time memory associations.
In LExMem benchmark experiments, LExGraph achieves better performance than existing retrieval-based and memory-based systems and generalizes effectively to external benchmarks.
LExMem poses a significant challenge to existing retrieval-based and memory-based systems.
All Facts (5)
Research from the MBZUAI NLP team proposes the graph-guided benchmark LExMem and the graph-based memory framework LExGraph for memory modeling and evaluation in long-horizon exploration tasks.
50%UnverifiedThe LExMem benchmark contains 224 dialogues and 2,018 question-answering instances, all derived from graph structures.
50%UnverifiedThe question-answer pairs in the LExMem benchmark are not manually written and then directly paired with context, but automatically derived from graph structures to ensure that questions depend on cross-branch and cross-time memory associations.
50%UnverifiedIn LExMem benchmark experiments, LExGraph achieves better performance than existing retrieval-based and memory-based systems and generalizes effectively to external benchmarks.
50%UnverifiedLExMem poses a significant challenge to existing retrieval-based and memory-based systems.
50%