Imitation Learning

模仿学习的核心思想是让机器人通过观察专家演示来习得行为策略，而非依赖大量试错，主要技术路线包括行为克隆（BC）和逆强化学习（IRL）

Behavioral cloning is the simplest form of imitation learning, directly learning a mapping from states to actions rather than inferring the expert's reward function as inverse reinforcement learning (IRL) does.

Behavior cloning transforms policy learning into a supervised learning problem, using environment states as inputs and expert actions as labels to train a mapping function from states to actions.

Modern AI robots typically rely on reinforcement learning to train motion control policies.

Compared with traditional reinforcement learning, imitation learning typically requires only tens to hundreds of demonstration trajectories for a robotic arm to learn grasping and placing, whereas reinforcement learning typically requires defining a reward function in a simulation environment and having the agent repeatedly trial and error, with sample sizes often in the millions.

Imitation Learning is a class of methods that lets robots learn policies directly from human demonstrations, mapping observations to actions through behavior cloning or diffusion policies, and its core advantage is high sample efficiency, but its out-of-distribution generalization ability is weak.

In robot demonstration collection, incorrect hand reconstruction will directly mislead the downstream imitation learning policy, causing the robot to learn actions that humans never performed, and errors in this kind of data itself will not be clearly exposed in the training loss.

Most existing methods for learning dexterous manipulation from video remain at the imitation level, strictly matching the action trajectories in the video, which leads to rapid failure when faced with new object initial poses, target poses, or grasp modes.

On the question of how humans align children, imitation learning may be more fundamental than reinforcement learning.

In imitation learning, data quality and demonstration consistency directly determine the performance of the policy.

FireFly's current training pipeline is primarily based on imitation learning/behavior cloning, training policies by recording screen observations and player actions.

The combination of imitation learning and reinforcement learning has transformed control policy development from writing rules one by one to learning from demonstrations.

Grok Bots' demonstration learning is technically close to a combination of program synthesis and imitation learning, automatically inferring reproducible operation scripts or tool-call chains by analyzing UI operation sequences from screen recordings.

Captured human motion data can theoretically be used for demonstration data collection in imitation learning, or as human prior trajectories to provide initial seeds for reinforcement learning policies

帧缺失导致的不连续跳跃会使模仿学习模型学到包含突变动作的策略，部署到真实机器人上时可能导致碰撞、振荡甚至硬件损坏

该方法属于模仿学习中的行为克隆，其著名缺陷是复合误差问题

该方法属于模仿学习范畴中最简单的形式——行为克隆，直接将专家演示视为有标签数据进行监督学习

模仿学习的代表性方法包括行为克隆、扩散策略以及ACT（Action Chunking with Transformers）