GUILD
A guided data collection framework for robot imitation learning that combines semantic priors from vision-language models with predictive data attribution, iteratively closing the loop to optimize the data acquisition strategy and reduce redundant data.
Timeline (last 90 days)
A research team at UT Austin proposed GUILD—a guided data collection method that actively reasons about "what data to collect" to maximize downstream policy performance.
GUILD alternates between two steps through an iterative framework: task proposal and data collection, and data evaluation.
In the task proposal stage, GUILD uses the semantic priors of a vision-language model (VLM) to identify relevant environments, object configurations, and target tasks.
GUILD introduces a data evaluation stage that uses predictive data attribution to approximately estimate the impact of collected trajectories on policy performance in the target deployment scenario.
The accuracy of predictive data attribution is constrained by the quality of the surrogate model, which is a key prerequisite for the reliability of GUILD's feedback signal.
The research team evaluated GUILD in simulation environments and on a real robot platform, and compared with passive data collection strategies, multiple baseline methods, and manual collection by experienced robot experts, guided data collection showed significant advantages in both sample efficiency and deployment performance.
Under the same data budget, GUILD can train better-performing policies faster and reduce the waste of expensive teleoperation time.
GUILD currently still depends on the quality of the VLM's semantic priors, and the accuracy of predictive attribution affects the effectiveness of the feedback signal.
In more complex open environments, the coverage of task proposals and the generalization ability of the attribution model still need further validation.
All Facts (9)
In more complex open environments, the coverage of task proposals and the generalization ability of the attribution model still need further validation.
50%UnverifiedGUILD alternates between two steps through an iterative framework: task proposal and data collection, and data evaluation.
50%UnverifiedIn the task proposal stage, GUILD uses the semantic priors of a vision-language model (VLM) to identify relevant environments, object configurations, and target tasks.
50%UnverifiedGUILD introduces a data evaluation stage that uses predictive data attribution to approximately estimate the impact of collected trajectories on policy performance in the target deployment scenario.
50%UnverifiedThe accuracy of predictive data attribution is constrained by the quality of the surrogate model, which is a key prerequisite for the reliability of GUILD's feedback signal.
50%UnverifiedThe research team evaluated GUILD in simulation environments and on a real robot platform, and compared with passive data collection strategies, multiple baseline methods, and manual collection by experienced robot experts, guided data collection showed significant advantages in both sample efficiency and deployment performance.
50%UnverifiedUnder the same data budget, GUILD can train better-performing policies faster and reduce the waste of expensive teleoperation time.
50%UnverifiedGUILD currently still depends on the quality of the VLM's semantic priors, and the accuracy of predictive attribution affects the effectiveness of the feedback signal.
50%UnverifiedA research team at UT Austin proposed GUILD—a guided data collection method that actively reasons about "what data to collect" to maximize downstream policy performance.
50%