Behavior Cloning
An imitation learning method that uses supervised learning to train an agent to directly replicate the action distribution demonstrated in human trajectories. It is a common technique for training GUI agents but is sensitive to distribution shift.
Core Facts
This experiment adopts trajectory-level distillation, which belongs to the category of behavioral cloning, directly having the student model imitate the external behavioral sequences of the teacher model.
Once incorrect experience learned by an agent from limited interactions becomes entrenched, it can lead to systemic bias and reduced work efficiency.
Behavioral cloning simplifies the problem to supervised learning, but suffers from distribution shift, where differences between the state distributions at training time and execution time lead to error accumulation.
Timeline (last 90 days)
Imitation Learning is a class of methods that lets robots learn policies directly from human demonstrations, mapping observations to actions through behavior cloning or diffusion policies, and its core advantage is high sample efficiency, but its out-of-distribution generalization ability is weak.
A common pathway for evolving from behavior cloning to reinforcement learning is to use the behavior cloning policy as a warm start initialization for RL followed by online fine-tuning.
Behavior cloning can only learn the state distribution covered by demonstration data, and error accumulation tends to occur when encountering rare situations.
FireFly's current training pipeline is primarily based on imitation learning/behavior cloning, training policies by recording screen observations and player actions.
Action sequences generated by human players completing Nuzlocke challenges can serve as expert demonstration data for imitation learning/behavioral cloning and offline reinforcement learning.
Many GUI agents are trained based on Behavior Cloning, where the model learns the action distribution from human demonstration trajectories rather than the essential goal of the task
Traditional behavioral cloning typically requires hundreds to thousands of demonstrations to train a usable policy model
Behavioral Cloning (BC) is the most straightforward implementation method in LfD, and is essentially a supervised learning problem
This experiment adopts trajectory-level distillation, which belongs to the category of behavioral cloning, directly having the student model imitate the external behavioral sequences of the teacher model.
Once incorrect experience learned by an agent from limited interactions becomes entrenched, it can lead to systemic bias and reduced work efficiency.
4 more timeline events
All Facts (14)
This experiment adopts trajectory-level distillation, which belongs to the category of behavioral cloning, directly having the student model imitate the external behavioral sequences of the teacher model.
75%VerifiedOnce incorrect experience learned by an agent from limited interactions becomes entrenched, it can lead to systemic bias and reduced work efficiency.
75%VerifiedBehavioral cloning simplifies the problem to supervised learning, but suffers from distribution shift, where differences between the state distributions at training time and execution time lead to error accumulation.
65%UnverifiedBehavior cloning transforms policy learning into a supervised learning problem, using environment states as inputs and expert actions as labels to train a mapping function from states to actions.
70%UnverifiedImitation Learning is a class of methods that lets robots learn policies directly from human demonstrations, mapping observations to actions through behavior cloning or diffusion policies, and its core advantage is high sample efficiency, but its out-of-distribution generalization ability is weak.
50%UnverifiedBehavior cloning can only learn the state distribution covered by demonstration data, and error accumulation tends to occur when encountering rare situations.
50%UnverifiedFireFly's current training pipeline is primarily based on imitation learning/behavior cloning, training policies by recording screen observations and player actions.
50%UnverifiedA common pathway for evolving from behavior cloning to reinforcement learning is to use the behavior cloning policy as a warm start initialization for RL followed by online fine-tuning.
50%UnverifiedAction sequences generated by human players completing Nuzlocke challenges can serve as expert demonstration data for imitation learning/behavioral cloning and offline reinforcement learning.
50%UnverifiedMany GUI agents are trained based on Behavior Cloning, where the model learns the action distribution from human demonstration trajectories rather than the essential goal of the task
50%UnverifiedTraditional behavioral cloning typically requires hundreds to thousands of demonstrations to train a usable policy model
50%UnverifiedBehavioral Cloning (BC) is the most straightforward implementation method in LfD, and is essentially a supervised learning problem
50%Unverified该方法属于模仿学习中的行为克隆，其著名缺陷是复合误差问题
50%Unverified该方法属于模仿学习范畴中最简单的形式——行为克隆，直接将专家演示视为有标签数据进行监督学习
50%