Persistent Retrieval Sub-agent
A retrieval architecture for multi-turn dialogue in which a sub-agent maintains retrieval history and state across turns and returns raw retrieved evidence directly to the main agent rather than generating a synthesized answer. The main agent handles evidence evaluation and final generation, decoupling retrieval from reasoning.
Timeline (last 90 days)
A study compared three conversational agent retrieval designs: skill-based main-agent retrieval, non-persistent retrieval sub-agent, and persistent retrieval sub-agent.
The core of the persistent retrieval sub-agent design is maintaining retrieval history across turns and returning retrieved evidence directly to the main agent, rather than having the sub-agent generate a synthesized response.
The persistent retrieval design allows the main agent to request corrected retrieval results from the same sub-agent.
Compared with the non-persistent retrieval sub-agent, the persistent design reduced response time per question by 15.6 seconds.
Compared with the non-persistent retrieval sub-agent, the persistent design used 65.1% more tokens and had an estimated API cost 29.1% higher.
Compared with skill-based main-agent retrieval, the persistent retrieval design used 43.3% fewer logged agent-model tokens, reduced estimated API cost by 45.0%, and cut average response time by 9.1%.
The study evaluated the three designs on enterprise questions, covering standalone questions, questions in shared conversations, and context-dependent follow-up questions, with evaluation spanning five multi-turn settings.
The persistent retrieval design outperforms skill-based main-agent retrieval in reducing token usage and API cost, while also improving response speed.
One limitation of the study is that the experiments focus on enterprise questions, and applicability to other domains (such as open-domain dialogue) remains to be verified.
The increase in token usage under the persistent design may pose challenges for some applications, requiring further optimization of methods for compressing and representing retrieval history.
1 more timeline events
All Facts (11)
A study compared three conversational agent retrieval designs: skill-based main-agent retrieval, non-persistent retrieval sub-agent, and persistent retrieval sub-agent.
50%UnverifiedThe core of the persistent retrieval sub-agent design is maintaining retrieval history across turns and returning retrieved evidence directly to the main agent, rather than having the sub-agent generate a synthesized response.
50%UnverifiedThe persistent retrieval design allows the main agent to request corrected retrieval results from the same sub-agent.
50%UnverifiedCompared with the non-persistent retrieval sub-agent, the persistent design reduced response time per question by 15.6 seconds.
50%UnverifiedCompared with the non-persistent retrieval sub-agent, the persistent design used 65.1% more tokens and had an estimated API cost 29.1% higher.
50%UnverifiedCompared with skill-based main-agent retrieval, the persistent retrieval design used 43.3% fewer logged agent-model tokens, reduced estimated API cost by 45.0%, and cut average response time by 9.1%.
50%UnverifiedThe study evaluated the three designs on enterprise questions, covering standalone questions, questions in shared conversations, and context-dependent follow-up questions, with evaluation spanning five multi-turn settings.
50%UnverifiedThe persistent retrieval design outperforms skill-based main-agent retrieval in reducing token usage and API cost, while also improving response speed.
50%UnverifiedOne limitation of the study is that the experiments focus on enterprise questions, and applicability to other domains (such as open-domain dialogue) remains to be verified.
50%UnverifiedThe increase in token usage under the persistent design may pose challenges for some applications, requiring further optimization of methods for compressing and representing retrieval history.
50%UnverifiedFuture research could explore dynamically adjusting retrieval strategies, combining the advantages of persistent and non-persistent approaches, and extending persistent retrieval to multimodal settings.
50%