CarryOn Dual-Loop Framework: A Personalized Robot Assistant That Actually Understands You

When a Robot Understands "Make Me a Sandwich"

General-purpose vision-language-action (VLA) models can already enable robots to complete a variety of manipulation tasks in the open world. But in reality, the robot assistants we need are far more than just "capable of doing work."

Imagine this: you tell the robot, "Make me a sandwich." A competent assistant needs to remember that you don't eat raw onions, prefer whole wheat bread, and want the sauce kept light—these preferences aren't one-off instructions, but personalized context that must be continuously tracked across conversations and scenarios. At the same time, the robot must ensure the process is safe and reliable, and if it knocks over a plate, it should be able to gracefully recover rather than standing there dumbly waiting for rescue.

These are precisely the two major shortcomings of current VLA models: personalization and reliable execution. The CarryOn framework from academia attempts to fill in these two pieces of the puzzle with a "dual-loop" architecture.

Vision-Language-Action (VLA) models are a class of foundation models that unify visual perception, natural language understanding, and robot action generation within a single neural network. Their fundamental difference from traditional robotic systems lies in this: traditional systems typically split perception, planning, and control into separate modules with hand-engineered interfaces, whereas VLA models directly map "the image seen + the instruction heard" to "where the joints should move," completing the entire task chain end-to-end. Representative works such as RT-2 and OpenVLA are typically pretrained on massive "image-text-action" triplet data, giving them a degree of open-world generalization capability—for example, understanding an instruction like "bring me that red thing" that never appeared exactly in training. But their limitations are also obvious: once training is complete, the model's behavioral patterns are frozen, making it difficult to personalize for a specific user's long-term preferences, and it lacks proactive risk anticipation and self-recovery from failures.

Dual-Loop Architecture: Division of Labor Between Planning and Execution

The core design idea of CarryOn is to layer two cooperating loops—a planning loop and an execution loop—on top of a "frozen" VLA policy. The reason for freezing the underlying VLA policy is that this allows flexible enhancement of system capabilities through external modules without retraining the large model.

Planning Loop: Remember Who You Are and Where You Are

The planning loop is responsible for maintaining two layers of persistent context: user-level and scene-level.

User-level context records personal preferences accumulated across conversations—such as dietary restrictions, operating habits, and requirements for speed or precision. Scene-level context tracks state changes in the current physical environment, such as what's on the kitchen counter and what ingredients remain in the fridge.

The key is that these contexts are not static database entries. The system continuously "grounds" existing knowledge with new observations and revises it through interaction. This means that when a user says "use the same brand as last time," the system can find the corresponding information from historical conversations rather than looking blank.

This design enables CarryOn to dispatch preference-aligned subtasks across multiple users and multiple scenes. For example, the same "prepare breakfast" instruction might be decomposed into "two sunny-side-up eggs + black coffee" for user A, but "oatmeal + juice" for user B.

"Grounding" in AI and robotics refers to the process of establishing correspondences between abstract symbols, language, or memories and concrete perceptual observations. For example, "the user doesn't like raw onions" is an abstract preference record, while "there are chopped red onions on the second shelf of the fridge" is a concrete visual observation; grounding is associating the former with the latter so that the system knows during actual operation that "this batch of onions should not go into the dish being made right now." In CarryOn's planning loop, grounding is continuous: whenever the robot receives new visual input or conversational feedback, the system recalibrates existing user-level and scene-level context to avoid memories becoming outdated due to environmental changes. This is similar to how a human assistant scans the counter each time they enter the kitchen, updating their mental model of "what ingredients are left" rather than mechanically following last week's shopping list.

Execution Loop: Safety Fallback and Graceful Recovery

The execution loop's role is more like that of a "gatekeeper." It actively monitors the VLA policy's action outputs and blocks actions predicted to involve dangerous contact.

The safety mechanism here is not simple collision detection, but proactive protection based on prediction. The system evaluates potential risks before action execution, intercepting operations that could lead to accidents in advance.

When execution does fail—such as a grasp slipping or an object tipping over—the system invokes reusable control-based recovery skills. These skills are not cobbled together on the fly, but refined through reflection on past failure cases, and can be reused in similar scenarios going forward.

Control-based recovery skills differ from end-to-end neural network policies in that the latter are black-box mappings that "generate actions directly from perception," while the former are deterministic policies hand-designed or solved via optimal control methods for specific physical interactions (such as regrasping, righting an object, or returning to a safe pose). The advantages of such skills are interpretability, reusability, and a high degree of parameterization—for example, a "regrasp after slip" skill can accept the pose of the target object as input and be called repeatedly across different scenarios. The reason CarryOn emphasizes "refined through reflection on past failure cases" is that each task failure records a failure mode (such as grasp point offset or insufficient friction), and the system adjusts the selection logic and parameters of recovery skills accordingly, so that next time a similar failure occurs, the correct recovery action can be invoked more quickly. This "failure-driven refinement" mechanism is essentially applying the experience replay idea from reinforcement learning to the iteration of a skill library.

The Data Speaks: Safety Violation Rate Drops from 52.6% to 6.1%

CarryOn has been validated on both real robots and in simulation environments, and several key numbers are worth noting:

• Safety violation rate: Even when the underlying VLA has already undergone safety fine-tuning, the violation rate remains as high as 52.6%. With CarryOn's execution loop added, this figure drops to 6.1%—a reduction of nearly 90%.
• Recovery success rate: When reusing refined recovery skills, the task recovery success rate jumps from 13% to 80%.

These two numbers illustrate an important point: relying solely on the VLA model's own safety fine-tuning is far from enough; external proactive supervision and structured recovery mechanisms are equally critical.

What This Framework Means

The value of CarryOn is not in proposing some entirely new VLA model, but in demonstrating a pragmatic engineering path: using architectural design to compensate for the shortcomings of foundation models.

For robot assistants moving toward real-world deployment, personalization makes users willing to use them, and reliable execution makes users dare to use them. CarryOn's dual-loop design targets both dimensions simultaneously, while maintaining the generality and transferability of the solution by freezing the underlying policy.

Of course, this framework also has its boundaries. The planning loop's context maintenance depends on high-quality interaction history, and the execution loop's recovery skill library needs to be accumulated in advance. How to reduce these dependencies and enable the system to grow autonomously in more open environments is a direction worth further exploration.

One-line Summary

CarryOn uses a planning loop and an execution loop to compensate for the shortcomings of VLA models in personalization and safety, reducing the safety violation rate to 6.1%.

Paragraph Summary

CarryOn is a robot framework that layers a dual-loop architecture on top of a frozen general-purpose vision-language-action (VLA) model, aiming to address two major shortcomings of current VLA models: cross-scenario personalization and reliable execution. The planning loop maintains persistent user-level and scene-level context, continuously grounding and revising it with new observations, and supports dispatching subtasks according to personal preferences; the execution loop proactively predicts and blocks dangerous actions, invoking reusable recovery skills refined through reflection when failures occur. In validation on real robots and in simulation, the safety violation rate dropped from 52.6% to 6.1%, and the recovery success rate rose from 13% to 80%. The core value of this framework lies in compensating for foundation model shortcomings through architectural design while maintaining generality and transferability.