FutureRTC: Real-Time Robot Execution with Anticipatory-Conditioned Action Chunking

1. Overview: Anticipatory-Conditioned Action Chunking for Real-Time Robot Execution

FutureRTC, proposed by Hai Jiang, Yixian Zou et al. in July 2026, is a plug-and-play adaptation framework that predicts execution-time observations and states to mitigate the prediction-execution misalignment in asynchronous VLA control, without modifying the underlying policy. Real-time VLA deployment requires asynchronous execution—subsequent action chunks are computed concurrently with current chunk execution, causing prediction-execution misalignment manifesting as inter-chunk discontinuities. Existing methods either superficially smooth chunk boundaries, require costly policy optimization, or exclusively forward-predict proprioceptive states while neglecting critical visual observations. FutureRTC features a State Correction Module (SCM) compensating the discrepancy between rolled-forward and actual execution-time proprioceptive states, and an Observation Prediction Module (OPM) forecasting execution-time visual representations by leveraging robot motion as an explicit physical prior through motion-aware feature transport and reconstruction. A policy consistency loss aligns action chunks generated from predicted contexts with those produced under expected execution-time inputs. Experiments demonstrate superior robustness to inference delays, smoother trajectories, faster execution, and higher success rates.

2. Core Problem: Prediction-Execution Misalignment in Asynchronous Execution

VLA models unify deep visual-linguistic reasoning with expressive action generation, but the tension between inference computational cost and high-frequency control demands gave rise to action chunking: the policy generates a sequence of future actions in a single forward pass, amortizing overhead through open-loop execution. Asynchronous execution became the standard paradigm for real-time deployment, but it introduces a fundamental prediction-execution misalignment—the policy generates actions at step $t+K-d$ conditioned on the stale pair $(o_{t+K-d}, s_{t+K-d})$ but executes them at step $t+K$ when the true observation and state have changed. Let the delay be $d$; the misalignment is:

$$\Delta_{\text{misalign}} = (o_{t+K}, s_{t+K}) - (o_{t+K-d}, s_{t+K-d})$$

As $d$ grows, the misalignment worsens sharply, causing performance collapse.

3. Method: Plug-and-Play Anticipatory Adapter

3.1 Overview

Given a flow-matching VLA $\pi_\theta(\cdot \mid o_t, s_t)$, the goal is to mitigate asynchronous execution misalignment where the policy is conditioned on the stale pair $(o_{t+K-d}, s_{t+K-d})$ prior to execution at step $t+K$. A plug-and-play anticipatory adapter $\phi(\cdot)$ forecasts execution-time observation and state from stale counterparts:

$$\hat{o}_{t+K}, \hat{s}_{t+K} = \phi(o_{t+K-d}, s_{t+K-d}, [a_{t+K-d}:a_{t+K-1}])$$

3.2 State Correction Module (SCM)

The execution-time state $s_{t+K}$ is unobservable at generation step $t+K-d$, but can be estimated by rolling forward the stale state using committed actions:

$$\tilde{s}_{t+K} = s_{t+K-d} \oplus \sum_{i=t+K-d}^{t+K-1} a_i \tag{3}$$

Forward integration inevitably accumulates bias due to the inherent deviation between commanded and actual robot motion. The SCM $\phi_{\text{SCM}}(\cdot)$ (MLP layers) predicts a compensatory residual:

$$\tilde{s}_\Delta = \phi_{\text{SCM}}\!\left(\tilde{s}_{t+K}, \frac{d}{d_{\max}}\right), \quad \hat{s}_{t+K} = \tilde{s}_{t+K} \oplus \tilde{s}_\Delta$$

The state correction loss minimizes the discrepancy between predicted residual and ground truth:

$$\mathcal{L}_{\text{state}} = \left\|(s_{t+K} \ominus \tilde{s}_{t+K}) - \tilde{s}_\Delta\right\|_2^2 \tag{4}$$

3.3 Observation Prediction Module (OPM)

SCM only mitigates part of the misalignment; observational staleness remains a critical unaddressed factor. The OPM $\phi_{\text{OPM}}(\cdot)$ leverages stale observations, committed actions, and corrected state to generate execution-time observations. Rather than reconstructing high-dimensional pixels, it operates in the latent space $z = \mathcal{E}(o)$ of the pretrained VLA's vision encoder $\mathcal{E}(\cdot)$. Since the primary visual variation is induced by the robot's physical displacement, feature evolution is modeled as spatial transportation rather than synthesis from scratch. For each committed action $a_i$ within the window, a physical feature vector is constructed:

$$\mathbf{u}_i = \left[a_i,\ \sum_{j \leq i} a_j,\ a_i - a_{i-1},\ \sum_{j \leq i} |a_j|,\ \frac{i}{d}\right] \tag{5}$$

encoding instantaneous command, cumulative displacement, action increment, accumulated path length, and temporal progress. Concatenated as $\mathbf{U} = [\mathbf{u}_1, \dots, \mathbf{u}_d]^\top$, fed into temporal self-attention to produce the action-conditioned motion prior $\mathbf{c}_{\text{motion}}$, fused with stale latent $\mathbf{z}_{t+K-d}$ and camera embedding via transport gate $\alpha$ and synthesis gate $\beta$ to generate predicted latent $\hat{z}_{t+K}$.

3.4 Policy Consistency Loss

To ensure predicted observation $\hat{z}_{t+K}$ and corrected state $\hat{s}_{t+K}$ are faithfully consumed by the policy, a policy consistency loss encourages the action chunk generated by the untouched policy from the anticipated pair $(\hat{z}_{t+K}, \hat{s}_{t+K})$ to align with that from the ground-truth pair $(z_{t+K}, s_{t+K})$. A single-step flow approximation avoids multi-step flow matching overhead:

$$\mathcal{L}_{\text{policy}} = \left\|\pi_\theta(z_{t+K}, s_{t+K}) - \pi_\theta(\hat{z}_{t+K}, \hat{s}_{t+K})\right\|_2^2 \tag{9}$$

The total objective is $\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{state}} + \mathcal{L}_{\text{obs}} + \lambda \mathcal{L}_{\text{policy}}$ with $\lambda=10$.

4. Experiments: Simulation and Real-World

4.1 LIBERO Simulation Benchmark

Comparing $\pi_{0.5}$ and SmolVLA-450M VLA backbones on LIBERO across delays $d \in \{5,10,15,20\}$:

Method	Category	$\pi_{0.5}$ d=5	$\pi_{0.5}$ d=20	SmolVLA d=5	SmolVLA d=20
Naive Async.	Infer.	89.5	68.3	70.3	56.2
TE	Infer.	87.8	74.8	68.4	59.7
RTC	Infer.	89.9	73.7	69.7	58.7
VLASH	Train.	88.0	74.5	69.3	61.9
REMAC	Train.	90.8	73.9	70.9	59.7
FutureRTC (Ours)	Infer.	94.2	88.5	75.8	69.4

FutureRTC leads across all delays. $\pi_{0.5}$ at d=20 reaches 88.5% (baseline Naive Async. 68.3%, +20.2pp); SmolVLA at d=20 reaches 69.4% (baseline 56.2%, +13.2pp).

4.2 Kinetix Dynamic Simulation

On Kinetix's 12 highly dynamic stochastic environments with prediction horizon $H=8$ and delay $d \in [0,4]$: Naive Async. collapses with delay; inference-time methods partially mitigate; training-time methods are stronger but still degrade under large delays due to reliance on stale observations. FutureRTC significantly outperforms all methods.

4.3 Real-World Evaluation

Deployed on a dual-arm AgileX Cobot Magic robot with $\pi_{0.5}$ backbone, execution horizon $K=25$, control frequency 30Hz ($\Delta t \approx 33$ms). A single VLA forward pass takes ~99ms; with LAN communication ~40ms and data processing ~30ms, end-to-end latency is ~170ms ($d=5$); injecting 150ms simulates remote deployment at $d=10$. Three bimanual tasks (Stack Plates, Fold Towel, Hang Cups), 20 trials each with randomized object poses.

4.4 Ablation Studies

Config	Description	Effect
Baseline	Direct stale pair	Rapid degradation with delay
+SCM	State correction only	Low-delay gains, ineffective at large delays
+OPM	Add observation prediction	Substantial improvements
+Policy consistency	Align with policy expectations	Overall further gains

This proves that stale visual observation—not just proprioceptive state—is the bottleneck of asynchronous execution.

5. Limitations

• VLA encoder dependency: OPM operates in the latent space of the pretrained VLA's vision encoder; if the encoder under-represents certain scenes, OPM is limited.
• Motion prior assumption: Assumes primary visual variation is from robot displacement; limited modeling capability for highly dynamic environmental changes (moving objects, lighting shifts).
• Single-step flow approximation: Policy consistency loss uses single-step flow approximation to avoid overhead, potentially introducing approximation error.
• Delay upper bound: Performance still degrades under extremely large delays exceeding the training $d_{\max}$.

6. Conclusion

FutureRTC is a plug-and-play adaptation framework that corrects rolled-forward proprioceptive states via SCM and predicts future visual representations via OPM using motion priors, enabling VLA policies to generate actions conditioned on execution-time observation and state, with a policy consistency loss ensuring predicted context is faithfully consumed. On LIBERO, Kinetix, and real bimanual manipulation, FutureRTC consistently improves delay robustness, execution smoothness, task efficiency, and success rates without modifying the underlying VLA policy. The core insight: rather than letting the policy decide from a stale snapshot, transport the stale observation to execution time using motion priors—visual staleness, not just state staleness, is the true bottleneck of asynchronous execution.

flowchart LR
    A["Stale obs o_t+K-d"] --> B["VLA vision encoder"]
    A2["Stale state s_t+K-d"] --> C["Forward roll + committed actions"]
    C --> D["Rolled state s̃_t+K"]
    D --> E["SCM correction residual"]
    E --> F["Corrected state ŝ_t+K"]
    G["Committed action window"] --> H["Physical feature u_i construction"]
    H --> I["Temporal self-attn → motion prior c_motion"]
    B --> J["Stale latent z_t+K-d"]
    J --> K["OPM: transport gate α + synthesis gate β"]
    I --> K
    F --> K
    K --> L["Predicted latent ẑ_t+K"]
    L --> M["VLA policy π_θ(ẑ_t+K, ŝ_t+K)"]
    F --> M
    M --> N["Execution-time action chunk"]