Smarter AI Doesn't Mean Smarter Robots — The Truth Is More Complicated

People often conflate robots and AI, as if they were the same project. But they're not — the AI research grabbing headlines (large vision models, reinforcement learning, large language models) mostly runs in simulated environments or data centers, while robots must contend with gravity, friction, battery drain, and the reality that a dropped object doesn't produce a loss function. So where exactly do the two intersect? And what are their respective bottlenecks?

Where AI Actually Touches Robots

A robot's software stack typically has three layers: perception, planning, and control. AI has made real progress in the first two.

The control layer — the part that translates plans into motor commands — is still dominated by classical methods: PID loops, model predictive control, inverse kinematics. AI sits on top of this layer rather than embedded within it. This layering explains a counterintuitive phenomenon: a robot can have a powerful vision model and still fail to reliably pick up a cup. The model tells it where the cup is, but not how much force the motors should apply before the cup slips.

PID loop (Proportional-Integral-Derivative control) is the most classic feedback control method: it continuously measures the error between a target value and the actual value, and outputs a correction command by weighting and summing three terms — the error's proportion, its accumulated sum, and its rate of change. It's nearly ubiquitous in robot joints, drone attitude control, and temperature control systems because it's computationally cheap, needs no training data, behaves predictably, and is easy to prove stable. Model predictive control (MPC) goes further: within each control cycle it solves a finite-horizon optimal control problem, predicts the system's response several steps ahead, picks the current optimal action, then rolls forward and repeats. MPC can explicitly handle constraints (torque limits, joint limits, obstacle avoidance), which is why it's become mainstream in quadruped gaits and autonomous driving trajectory tracking. Inverse kinematics solves the opposite problem: given a desired pose for the end effector, it works backward to find the joint angles required. Together, these three methods form the backbone of the control layer — and explain why no matter how powerful the AI model above, it can't directly dictate motor current: that's the result of the control layer's repeated low-level closed-loop corrections.

Vision Models: See First, Act Second

Vision is AI's most mature contribution to robotics. Object detection and segmentation models let robots find items on shelves, identify surgical landmarks, and track packages on conveyor belts. Depth estimation — usually from stereo cameras or LiDAR fused with learned models — gives robots a sense of three-dimensional space rather than a flat image.

# Simplified perception loop
frame = camera.capture()
detections = vision_model.predict(frame)
for obj in detections:
    depth = depth_estimator.estimate(obj.bbox, frame)
    world_pose = camera_to_world(obj.bbox, depth)
    planner.add_target(world_pose)

This loop runs several times per second on a real robot. The model's job ends at generating poses. Everything after — approaching the object and grasping it with the right force — is a separate problem.

Reinforcement Learning: Learning to Move

Reinforcement learning has trained a wide range of locomotion and manipulation policies. Rather than hand-coding a gait, you run a robot thousands of times in simulation, reward staying upright and moving forward, and let the policy emerge on its own.

**Sim-to-real transfer** refers to the performance drop when policies trained in simulators are deployed on real hardware. The root cause is that simulation can't perfectly reproduce real physics: friction coefficients change with temperature, motors have backlash and latency, sensors have noise and drift, and contact forces are hard to model precisely. Common mitigations include **domain randomization** — randomly perturbing mass, friction, latency, lighting, and other parameters during training to force the policy to be robust to them; **system identification** — calibrating simulator parameters with real data; and **real-world fine-tuning** — continuing training on hardware with a small amount of data. Even so, no method fully closes the gap, which is why many gait policies that perform flawlessly in simulation fall over the moment they hit real hardware. Transfer quality often determines whether a robotics project stays in a paper or makes it out of the lab.
# Simplified RL training loop
for episode in range(num_episodes):
    state = env.reset()
    done = False
    while not done:
        action = policy(state)
        next_state, reward, done = env.step(action)
        buffer.store(state, action, reward, next_state)
        state = next_state
    policy.update(buffer.sample())

Boston Dynamics' Atlas and Tesla's Optimus both rely on this kind of training for balance and recovery. Sim-to-real transfer is the hard part: policies that work in simulation often need real-world fine-tuning, because friction, motor backlash, and sensor noise can never perfectly match the simulator.

Robots and Mobile Platforms Already Deployed Today

Setting aside the lab, AI is already driving robots to create real value in several scenarios:

• Warehouse logistics: Amazon's fleet of mobile robots moves shelving and packages through fulfillment centers, with AI path planning that dynamically avoids people and obstacles.
• Humanoid robots: Figure's robots and Tesla's Optimus are being tested on tasks like sorting and simple assembly, using vision-language-action models to map camera feeds and voice commands into action sequences.
• Surgical robots: Intuitive Surgical's da Vinci system still relies on human surgeons for decisions, but AI assists with instrument tracking and tissue recognition.
• Autonomous drones: AI-driven drones are already used for power line inspection, crop monitoring, and search-and-rescue, using onboard vision models to avoid obstacles when GPS is unreliable.

What LLMs Add

LLMs are entering robotics as planners, not controllers. Given a high-level instruction like "clean the table," an LLM can break it into subtasks (locate objects, pick them up one by one, place them in a box), then hand them off to low-level motion policies. Google's RT-2 and similar vision-language-action models go further, mapping images and text directly to robot actions. The LLM doesn't touch any motors — it decides what should happen next and leaves execution to the control stack described earlier.

Vision-language-action models (VLA) are a class of models that emerged after 2023, with representative work including Google's RT-2 and the open-source OpenVLA. The core idea is to treat robot actions as a kind of "language": feed image frames and text instructions into a pretrained multimodal large model, which directly outputs discretized action tokens that a low-level controller then executes. Compared to the traditional perceive-plan-control pipeline, VLA eliminates hand-designed intermediate representations, letting the model learn end-to-end mappings from "what it sees and hears" to "what action to take." Its strengths are strong generalization and the ability to understand unseen objects and instructions; its limitations are low inference frequency (typically only a few hertz) and action precision that falls short of specialized controllers, so it's mostly used for relatively coarse-grained tasks like grasping and placing, while fine manipulation still needs classical control to fill the gap.

The Real Limits

AI progress doesn't automatically eliminate the physical world's constraints:

• Power: Humanoid robots like Optimus run about 2–5 hours on moderate work, far less than a human shift.
• Dexterity: The human hand has 27 degrees of freedom plus touch; current robot grippers still can't match that, which is why most warehouse robots carry boxes rather than individual items.
• Cost: A force-sensing manipulator arm still costs tens of thousands of dollars, keeping humanoid robots out of most workplaces.
• Safety: Robots sharing space with humans need certified force limits and fail-safe mechanisms, which slow deployment more than the AI models themselves.

Degrees of Freedom (DoF) refers to the number of directions in which a mechanical system can move independently. The human hand has 27 degrees of freedom, plus densely distributed tactile receptors, enabling it to do things like turn screws, pinch eggs, and tie shoelaces — movements that require fine force feedback. Today's mainstream two- or three-finger grippers typically have only 1 to 6 degrees of freedom with sparse tactile sensing, unable to feel whether an object is about to slip and relying only on rough visual estimation. This directly explains why warehouse robots tend to carry entire boxes rather than pick up items one by one — boxes have regular shapes and clear grasp points, while arbitrarily shaped single items require human-hand-level dexterity. More degrees of freedom means more motors, more complex transmissions, and higher cost — which is exactly why dexterous hands remain confined to labs and high-priced products.

What Comes Next

The next few years look less like "robots getting smarter" and more like "robots getting cheaper and more reliable at narrow tasks." AI will keep improving perception and task planning, while mechanical engineering, battery density, and manufacturing cost close the gap from the physical side. The bottleneck has shifted: it used to be software, and increasingly it's the hardware beneath the software.

One-line Summary

AI only helps robots at the perception and planning layers; the control layer remains classical, and the bottleneck has shifted from software to hardware.

Paragraph Summary

The article clarifies the misconception that robots and AI are the same thing: the AI research grabbing headlines mostly runs in simulation or data centers, while robots must face physical realities like gravity, friction, batteries, and dropped objects. A robot's software stack has three layers — perception, planning, and control — and AI has made real progress in the first two, while the control layer is still dominated by classical methods like PID, MPC, and inverse kinematics. Vision models let robots see and locate objects, reinforcement learning trains the locomotion policies of Atlas and Optimus, but sim-to-real transfer remains difficult. LLMs enter robotics as planners rather than controllers, and vision-language-action models like RT-2 map images and text directly to actions. The article also points out four physical bottlenecks — power, dexterity, cost, and safety — and concludes that the future trend is cheaper, more reliable robots for narrow tasks.