ROMA
A large language model-based multi-sensory active perception system that integrates visual, tactile, and other sensing modalities around object-centric representations. The LLM serves as a planning core to actively identify information gaps, acquire relevant data, and evaluate evidence sufficiency, enabling closed-loop perception and decision-making on real hardware.
Timeline (last 90 days)
ROMA is a real-world object-centric multi-sensory active perception system that enables the system to actively determine what information is currently missing, purposefully acquire that information, and continuously track whether the evidence is sufficient to make a final judgment about object properties.
The core mechanisms of ROMA include three interlocking capabilities: identifying information gaps, actively acquiring information, and evaluating evidence sufficiency.
ROMA delegates the three decisions of 'when to explore, what to explore, and when to stop' entirely to language model reasoning, rather than hard-coding them as fixed rules.
ROMA is built on a large language model (LLM) planner, where the LLM is responsible for understanding task goals, inferring the current information state, and generating the next information-acquisition action.
ROMA is built on real multi-sensory hardware rather than remaining in a simulated environment, validating its active perception closed loop in real-world scenarios.
ROMA has simultaneously released both the complete paper and open-source code.
ROMA's evidence sufficiency evaluation mechanism can avoid resource waste caused by excessive data collection, as well as the risk of drawing hasty conclusions with insufficient evidence.
All Facts (7)
ROMA is built on a large language model (LLM) planner, where the LLM is responsible for understanding task goals, inferring the current information state, and generating the next information-acquisition action.
50%UnverifiedThe core mechanisms of ROMA include three interlocking capabilities: identifying information gaps, actively acquiring information, and evaluating evidence sufficiency.
50%UnverifiedROMA delegates the three decisions of 'when to explore, what to explore, and when to stop' entirely to language model reasoning, rather than hard-coding them as fixed rules.
50%UnverifiedROMA is a real-world object-centric multi-sensory active perception system that enables the system to actively determine what information is currently missing, purposefully acquire that information, and continuously track whether the evidence is sufficient to make a final judgment about object properties.
50%UnverifiedROMA is built on real multi-sensory hardware rather than remaining in a simulated environment, validating its active perception closed loop in real-world scenarios.
50%UnverifiedROMA's evidence sufficiency evaluation mechanism can avoid resource waste caused by excessive data collection, as well as the risk of drawing hasty conclusions with insufficient evidence.
50%UnverifiedROMA has simultaneously released both the complete paper and open-source code.
50%