Chronos Policy: Teaching Robots to Perceive Real Physical Time

When Robot Policies Ignore Time

Most mainstream robot policies treat perception and control as a fixed-frequency sequence of observation-action pairs, where physical time is implicitly encoded in the positional order of the sequence. In other words, the model assumes a constant time interval between steps — time information is hidden in sequence indices rather than being explicitly modeled.

This assumption works well under ideal conditions, but real-world deployment is far messier. Observation frequencies may fluctuate, sensors can have varying delays, and inference latency shifts with system load. Once these timing conditions deviate from what was seen during training, policies with implicit time assumptions fall apart — because the model simply doesn't know "how long has passed since the last observation."

Chronos Policy directly addresses this pain point. Its core argument is straightforward: make physical time an explicit input to the policy, so the model knows when each observation was captured and when each action is expected to be executed.

Two Ways to Inject Time into a Diffusion Model

Chronos Policy is built on a diffusion Transformer architecture. The researchers explored two approaches for incorporating physical time into the model.

Timestamp Embeddings vs. Time-Aware RoPE

The first approach adds timestamp embeddings directly on top of token features — essentially attaching a "time tag" to each token. The second approach is more elegant: it encodes physical time into the positional encoding itself via time-aware Rotary Position Embedding (RoPE).

Experiments show that time-aware RoPE is more robust. Its advantage lies in allowing the attention mechanism to compute relationships between tokens based directly on physical time, while keeping token content uncontaminated. This means the model naturally accounts for time intervals when deciding "who should attend to whom," rather than forcing time information into semantic features.

This design tradeoff is instructive: rather than simply expanding feature dimensions, letting time act on the geometric structure of attention tends to yield cleaner, more transferable representations.

---

Rotary Position Embedding (RoPE) was originally proposed by Su et al. as a replacement for traditional absolute position encodings in Transformers. The core idea: instead of adding positional information directly to token feature vectors, RoPE applies a rotation transformation to the Query and Key vectors in the attention computation, so that the attention score between two tokens depends only on their relative position, not their absolute positions. This property gives RoPE natural extrapolation capability and makes it easier to generalize to continuous, non-integer position indices.

Time-aware RoPE exploits exactly this property: it replaces the integer index representing "the i-th token in the sequence" with the actual physical timestamp of sensor capture or action execution (e.g., a floating-point number in seconds). The attention mechanism can then directly perceive how much time has elapsed between two observations — a 0.1-second gap versus a 1-second gap produces geometrically different rotation angles, which in turn affects how information is aggregated. By contrast, concatenating timestamps to token feature vectors forces the model to additionally learn "how to interpret time differences from semantic space," which corrupts the semantic structure of the features and makes it harder to generalize to time intervals not seen during training.

---

Temporal Randomization During Training

Another key design choice in Chronos Policy lies in the training procedure. The researchers apply temporal randomization in two ways: randomly sampling which observations are used, and randomly deciding when the action chunk begins execution.

The goal is to expose the policy during training to the variety of perception and execution rhythms it will encounter at deployment, rather than always seeing a single fixed timing pattern. The model is thus forced to learn stable outputs under diverse timing conditions, improving generalization to temporal variation.

This mechanism also brings an additional benefit: it enables the policy to learn from asynchronous, multi-rate data without needing to force-align all measurements onto a fixed time grid. For systems where sensors naturally operate at different rates (e.g., high-frequency force sensing paired with low-frequency vision), this flexibility is especially important — the interpolation or frame-dropping that used to be required for alignment is itself often a source of error.

---

An action chunk is a common output format in current diffusion-based policies, where the model predicts a sequence of future actions all at once rather than outputting one action at a time. This design originates from works like ACT and Diffusion Policy; its benefit is reducing the accumulated error of autoregressive step-by-step inference while allowing the model to plan over short motion trajectories holistically. However, action chunks carry an implicit temporal convention: each action in the chunk is assumed to execute at fixed, equally spaced intervals. Once the actual execution rhythm shifts due to inference latency or system jitter, the entire chunk's temporal meaning is misaligned. Chronos Policy addresses this by randomizing the starting execution time of action chunks during training, forcing the model to learn conditional generation of the form "given that this chunk should start executing at time t" — rather than assuming execution always begins at some fixed offset. This fundamentally eliminates the brittleness introduced by that implicit convention.

---

Experimental Results: Gains on Dynamic and Multi-Sensor Tasks

Across simulation and real-world experiments, Chronos Policy demonstrates improvements at several levels.

On dynamic tasks, where the timing between observation and action is critical, Chronos Policy delivers clear performance gains. In these tasks, when to move matters just as much as what to do — explicitly modeling time translates directly into measurable improvements.

On multi-sensor tasks, where the relative timing between different observations becomes a key variable, Chronos Policy also performs better. When multiple sensors provide information at different rates, knowing each sensor's timestamp is essential for correctly fusing those signals.

On standard quasi-static tasks, where timing has limited impact, Chronos Policy maintains performance on par with the baseline while improving robustness to variations in perception and inference timing. This is important: it shows that explicitly modeling time does not sacrifice performance on common scenarios in order to gain on dynamic ones. It is an incremental design that is "better when needed, harmless when not."

---

Quasi-static tasks are a common benchmark category in robot manipulation research, referring to interactions where changes happen slowly, objects remain largely stationary, and the precise timing of actions has limited effect on outcomes — typical examples include grasping and placing objects on a table, opening drawers, and peg insertion. These tasks are the primary validation setting for early imitation learning and diffusion policy papers, which is precisely why the implicit time assumptions of existing policies rarely cause visible problems there. By contrast, dynamic tasks (e.g., catching a moving object, working with a conveyor belt) and multi-modal sensing tasks (e.g., fusing vision and force feedback) demand higher timing precision and serve as more rigorous tests of temporal modeling capability. The fact that Chronos Policy maintains performance on quasi-static tasks confirms that explicit time modeling is a "fix weaknesses" rather than a "rob Peter to pay Paul" tradeoff — an important reliability guarantee for practitioners who want to deploy the same policy framework across different task types.

---

Why This Work Deserves Attention

Chronos Policy addresses a long-underestimated problem in robot learning: the implicit assumption about time. As robot policies are increasingly deployed in real, uncontrolled environments, the idealized conditions of fixed frequency and fixed latency essentially never hold. Inference latency fluctuates, sensors are asynchronous — these are facts of deployment, not the lab.

Freeing physical time from its role as a sequence position index and making it an explicit quantity the model can reason about is a meaningful step toward robust real-world deployment. Time-aware RoPE combined with temporal randomization during training forms a relatively lightweight but targeted solution — one that doesn't require redesigning the entire architecture while delivering cross-task generalization improvements.

For practitioners focused on embodied intelligence and robot foundation models, this work offers a clear insight: when a policy must face real-world temporal uncertainty, rather than letting the model implicitly guess, just tell it what time it is.