GUILD: Improving Imitation Learning Efficiency with Guided Data Collection

Robot imitation learning typically relies on expensive human teleoperation data collection, and unguided collection often produces large amounts of redundant data that contribute little to policy performance. Recent research shows that filtering existing datasets can improve policy performance, but this is only a post-hoc remedy that cannot prevent the generation of ineffective data. To address this, a research team from UT Austin proposes GUILD—a Guided Data Collection (GDC) method that actively reasons about "what data to collect" to maximize downstream policy performance.

From Passive Filtering to Active Guidance

Traditional imitation learning data collection pipelines rely heavily on operator intuition, resulting in trajectories of varying quality. Although data filtering methods can remove low-quality samples, filtering itself does not change the blindness of the collection stage—ineffective data is still collected, and the cost has already been incurred. The core insight of GUILD is to treat data collection as an optimizable closed-loop process rather than a one-time action. It alternates between two steps through an iterative framework: task proposal and data collection, and data evaluation. This closed-loop design allows the collection strategy to dynamically adjust based on evaluation feedback, thereby reducing the generation of useless data at the source.

Imitation Learning is one of the mainstream paradigms for robot skill acquisition. Its basic setup is: a human expert demonstrates a task through teleoperation or other means, and the resulting state-action sequences are used as supervision signals to train a policy network. Unlike reinforcement learning, which relies on reward signals and trial-and-error, imitation learning learns directly from demonstrations, resulting in higher sample efficiency and more stable training. However, the cost is that collecting demonstration data itself is extremely expensive—operators need to repeatedly execute tasks, and trajectory quality depends heavily on operator skill.

Teleoperation refers to an operator remotely controlling a robot through controllers such as joysticks, master-slave arms, or VR devices to complete actions, with the system synchronously recording sensory data such as joint angles, end-effector poses, and images. It is currently the main source of robot demonstration data, but collection speed is usually far lower than the robot's actual execution speed, and prolonged operation easily leads to fatigue and decreased consistency. This explains why data collection cost has become the core bottleneck for scaling imitation learning, and it also constitutes the object that GUILD attempts to optimize.

Synergy of Semantic Priors and Predictive Attribution

In the task proposal stage, GUILD leverages the semantic priors of vision-language models (VLMs) to identify relevant environments, object configurations, and target tasks. For example, the model can infer which scene combinations are more likely to produce information gain for the target policy. However, semantic relevance does not equate to policy performance improvement—a semantically plausible scene may not provide the most valuable gradient signals for policy learning. Therefore, GUILD introduces a data evaluation stage that uses predictive data attribution to approximately estimate the impact of collected trajectories on policy performance in the target deployment scenario. This attribution signal is fed back to the collection stage to guide the direction of the next round of data collection.

Vision-Language Models (VLMs) are models pretrained on large-scale image-text pairs that can jointly understand image content and natural language descriptions. Typical capabilities include open-vocabulary object recognition, scene relationship reasoning, and commonsense judgment. In robotics, VLMs are often used as a source of "semantic priors" to determine whether an object or scene configuration is relevant to the target task. Their advantage is that they can provide usable commonsense reasoning without task-specific retraining; their limitation is that their judgments remain at the semantic level and cannot directly reflect the actual contribution of a sample to policy gradients or parameter updates.

Data Attribution is a class of methods in machine learning that quantify "the impact of an individual training sample on model output or performance." Common approaches include influence functions, leave-one-out approximations, and Shapley value-based contribution allocation. Predictive data attribution further introduces a surrogate model to predict how much performance change a trajectory would bring if added to the training set, without actually training the target policy on that data. This approximation avoids the high cost of repeated retraining, but it also means that attribution accuracy is constrained by the quality of the surrogate model—which is precisely the key prerequisite for the reliability of GUILD's feedback signal.

Experimental Validation: Simulation and Real Robots

The research team evaluated GUILD in simulation environments and on real robot platforms. Compared with passive data collection strategies, multiple baseline methods, and manual collection by experienced robot experts, guided data collection demonstrated significant advantages in both sample efficiency and deployment performance. This means that under the same data budget, GUILD can train better-performing policies faster and reduce the waste of expensive teleoperation time. For robot learning, data collection cost has always been a bottleneck for scaling and deployment, and GUILD provides a systematic technical path to reduce this cost.

Technical Significance and Limitations

The value of GUILD lies in transforming "data collection" from an engineering step that relies on human intuition into a computable, optimizable algorithmic problem. It combines the semantic understanding capabilities of VLMs with the quantitative evaluation of data attribution to form an iterative active learning framework. However, the method still depends on the quality of VLM semantic priors, and the accuracy of predictive attribution affects the effectiveness of the feedback signal. In addition, in more complex open environments, the coverage of task proposals and the generalization ability of the attribution model still require further validation.

For more information, visit the project page: https://robin-lab.cs.utexas.edu/guild2026/

One-line Summary

UT Austin proposes GUILD, which uses VLM semantic priors and data attribution to actively guide robot data collection.

Paragraph Summary

Robot imitation learning relies on expensive human teleoperation data, and blind collection produces large amounts of redundant samples that contribute little to policy performance; post-hoc filtering can only remedy this but cannot prevent ineffective data from being generated. A research team from UT Austin proposes GUILD, a guided data collection method that treats data collection as an optimizable closed-loop process, alternating between two steps: task proposal and data collection, and data evaluation. The task proposal stage uses vision-language model semantic priors to identify relevant environments, object configurations, and target tasks; the data evaluation stage uses predictive data attribution to approximately estimate the impact of collected trajectories on policy performance in deployment scenarios, and feeds this signal back to guide the next round of collection. In simulation and on real robot platforms, GUILD outperforms passive collection, multiple baseline methods, and expert manual collection in both sample efficiency and deployment performance, providing a systematic path to reduce the data cost of robot learning.