HiVe-Mem
A memory framework for multimodal agents that improves the coverage of memory retrieval and the efficiency of evidence use in long-term interaction through two dimensions: horizontal memory expansion (neighbor expansion based on a relational graph) and vertical evidence composition (query-adaptive selection of evidence subsets).
Timeline (last 90 days)
HiVe-Mem is a multimodal memory framework that interleaves horizontal memory expansion with vertical evidence composition.
HiVe-Mem organizes memory segments into a relational graph, with nodes connected through textual anchors and visual anchors.
Horizontal expansion refers to that, after semantic retrieval hits initial seed memories, the system expands along the graph structure to their neighboring nodes, bringing semantically related memories that were not directly retrieved into the candidate set.
In the vertical dimension, HiVe-Mem lets each retrieved memory retain multiple forms of evidence, including compact summaries, descriptive text, visual-grounding-based views, raw images, and interaction history.
A lightweight trainable selector independently composes an evidence subset for each memory, thereby adaptively exploiting heterogeneous multimodal information according to the specific query.
The researchers conducted experiments on three multimodal memory benchmarks, and the results show that HiVe-Mem can consistently improve task performance across different base models.
Compared with the most efficient competitive multimodal baseline, HiVe-Mem reduces token consumption by about 90%.
Horizontal expansion broadens the recall coverage of memories, while vertical composition optimizes how each memory's evidence contributes, and the two work in concert.
All Facts (8)
HiVe-Mem is a multimodal memory framework that interleaves horizontal memory expansion with vertical evidence composition.
50%UnverifiedHiVe-Mem organizes memory segments into a relational graph, with nodes connected through textual anchors and visual anchors.
50%UnverifiedHorizontal expansion refers to that, after semantic retrieval hits initial seed memories, the system expands along the graph structure to their neighboring nodes, bringing semantically related memories that were not directly retrieved into the candidate set.
50%UnverifiedIn the vertical dimension, HiVe-Mem lets each retrieved memory retain multiple forms of evidence, including compact summaries, descriptive text, visual-grounding-based views, raw images, and interaction history.
50%UnverifiedA lightweight trainable selector independently composes an evidence subset for each memory, thereby adaptively exploiting heterogeneous multimodal information according to the specific query.
50%UnverifiedThe researchers conducted experiments on three multimodal memory benchmarks, and the results show that HiVe-Mem can consistently improve task performance across different base models.
50%UnverifiedCompared with the most efficient competitive multimodal baseline, HiVe-Mem reduces token consumption by about 90%.
50%UnverifiedHorizontal expansion broadens the recall coverage of memories, while vertical composition optimizes how each memory's evidence contributes, and the two work in concert.
50%