$π\mathbf{R}^2$: Reactive Real-time Flow Policies

Paper: πR²: Reactive Real-time Flow Policies
Authors: Sungjae Park, Shubham Tulsiani (Carnegie Mellon University)
Links: arXiv 2607.26055 (2026-07-28, cs.RO) | Project page
Code status: Official code is open-sourced at pi-r2-flow/pi-r2-flow, including the xArm6 + XHand deployment stack and an Isaac-GR00T training fork.

One-Sentence Summary

πR² turns action-chunking flow policies into reactive real-time controllers by separating fresh proprioception from asynchronously cached vision-language features and using a latency-adaptive noise schedule that emits actions with one denoising step per call. Fine-tuned from GR00T-N1.7 on a real xArm6 + XHand setup, it replans at 25 Hz with observations no older than 40 ms and improves success by up to 23% in simulation and 30% in the real world.

Background and Motivation

Generalist manipulation policies now routinely combine three design choices: a large pretrained vision-language backbone, an expressive diffusion or flow-matching action head, and action chunking that predicts multiple future actions together. This recipe has made imitation learning more scalable, but it also creates a fundamental tension with closed-loop control: the policy commits to a chunk and cannot react to sensory input that arrives while that chunk is being executed.

The obvious remedy is to replan more often, but the perception-to-action pipeline is too slow. On an RTX A6000 with GR00T-N1.7, image preprocessing plus VLM forward takes roughly 60 ms, and four DiT denoising steps take roughly 80 ms. A single call therefore costs about 140 ms, or about seven control ticks at 50 Hz. In practice, a large VLA replans only sparsely, and the actions it commits to are conditioned on stale observations.

Diffusion forcing offers a promising alternative because it assigns an independent noise level to every position in an action chunk. The leading positions can be kept near clean and generated in fewer steps, and successive denoising steps can use progressively fresher observations. Prior streaming policies exploited this idea, but they mostly used compact visuomotor networks and synchronous pipelines. In a large VLA, the expensive backbone forward remains the bottleneck even if the action head becomes cheaper.

The key insight behind πR² is that not all input modalities need to be refreshed at the same rate. Proprioception can be read and processed orders of magnitude faster than images or text, and it carries exactly the local signals needed for reactive corrections: unexpected contact, object slip, force overshoot, or a book landing in the palm. Vision and language provide coarse spatial and task guidance, and they can tolerate bounded staleness. The paper turns this asymmetry into an architectural split.

The paper makes two contributions. First, proprioception-reactive diffusion forcing splits conditioning into a fast channel updated every tick and a slow vision-language channel updated asynchronously. Second, a latency-adaptive flow schedule treats in-flight actions as inpainting conditioning and emits $d$ clean actions per call from a single denoising step. Together, they reduce per-call delay by up to 4x without discarding large backbones, multimodal policies, or multi-action prediction.

Preliminaries

Flow matching learns a velocity field $v_{\theta}(\mathbf{x}_{t},t)$ that transports noise from $p_{0}=\mathcal{N}(\mathbf{0},\mathbf{I})$ to data $p_{1}=p_{\mathrm{data}}$ along the conditional interpolation

$$\mathbf{x}_{t}=(1-t)\boldsymbol{\epsilon}+t\mathbf{x}_{1},\quad t\in[0,1]$$

with conditional velocity $u_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{1})=\mathbf{x}_{1}-\boldsymbol{\epsilon}$. The training objective minimizes the mean squared error between the network velocity and this conditional velocity:

$$\mathcal{L}_{\mathrm{FM}}=\mathbb{E}_{t,\mathbf{x}_{1},\boldsymbol{\epsilon}}\big[\|v_{\theta}(\mathbf{x}_{t},t)-(\mathbf{x}_{1}-\boldsymbol{\epsilon})\|^{2}\big]$$

In action-chunking policies, $\mathbf{x}_{1}=(\mathbf{a}_{t},\dots,\mathbf{a}_{t+H})$ is a chunk of $H$ future actions generated by $K$ denoising steps. The robot executes the first $h\leq H$ actions before replanning. Because standard flow matching uses one shared noise level $t$ for all positions, the whole chunk is generated under one observation and executed open-loop.

Diffusion forcing generalizes this schedule by assigning every chunk position $p$ an independent noise level $\tau_{p}$:

$$\mathbf{x}_{\tau,p}=(1-\tau_{p})\boldsymbol{\epsilon}_{p}+\tau_{p}\mathbf{a}_{p}$$

The model $v_{\theta}(\mathbf{x}_{\tau},\boldsymbol{\tau},\mathbf{o})$ predicts a per-position velocity. Streaming diffusion uses an increasing noise schedule so leading actions become clean after a few steps, and each step can consume a fresh observation. πR² builds on this machinery but targets the regime where a large pretrained backbone makes repeated full-model calls impractical.

Method

Figure 1: πR² overview. Top shows the observation split into a fast proprioception channel and a slow vision-language channel; bottom shows the three-region staircase with in-flight actions, an interior ramp, and fresh tail noise.

Fast Proprioception, Slow Vision-Language Features

πR² represents the observation as $(\mathbf{s}_{t},\mathbf{I}_{t},\mathbf{T}_{t})$: proprioception, image, and language. The standard VLA preprocesses all inputs, passes them through the VLM, and conditions a DiT action head on the concatenated representation. The paper separates the DiT conditioning into two channels, with the action loop using fresh proprioception every control tick and a cached slow feature supplied by a background VLM worker.

To make the action head robust to slow-channel staleness, training delays the slow feature by $d_{\mathrm{vlm}}\sim\mathrm{Uniform}\{0,\dots,d_{\mathrm{vlm}}^{\max}\}$ ticks. A learned lookup table $e(d_{\mathrm{vlm}})$ encodes the integer delay and is added to the slow representation. At deployment, the measured wall-clock vision latency uses the same embedding, so the network knows the age of the cached visual feature rather than assuming it is fresh.

This split mirrors the structure of manipulation: vision and language anchor the task and scene, while fresh proprioception drives contact, slip, and force refinement. In simulation, a 9-dimensional vision-derived subset plays the slow-channel role. In the real GR00T-N1.7 deployment, the VLM cache is refreshed continuously on a separate GPU while the action head performs a lightweight DiT forward per tick.

Latency-Adaptive Staircase Schedule

Even with one denoising step per call, measured latency $d$ varies across hardware, network conditions, and load. πR² defines a three-region staircase $\boldsymbol{\tau}^{\star,d}$ for a target delay $d$:

$$\tau_{p}^{\star,d}=\begin{cases}1 & 0\le p<d\\[2pt] 1-\frac{p-d}{H-2d} & d\le p<H-d\\[2pt] 0 & H-d\le p\le H-1\end{cases}$$

The front region $[0,d)$ holds the in-flight actions as clean inpainting conditioning. The interior $[d,H-d)$ is a linear ramp from clean to noise. The tail $[H-d,H)$ appends fresh noise slots that enter after each slide. Unlike training-time RTC, which uses one shared noise level over the remaining positions, the ramp allows one Euler step to release the front of the interior as clean actions.

Training samples $d\sim\mathrm{Uniform}\{1,\dots,d_{\mathrm{max}}\}$ and builds $\boldsymbol{\tau}^{\star,d}$. Ground-truth actions fill the front $d$ slots, and the loss masks them out with $m_{p}=\mathbf{1}[p\geq d]$. The interior and tail contribute per-position MSE. Symmetric jitter $\tau_{p}\leftarrow\mathrm{clip}(\tau_{p}+\delta_{p},0,1)$ with $\delta_{p}\sim\mathrm{Uniform}[-j,j]$ absorbs small deviations from the central schedule.

With probability 0.2, training instead uses a standard flow schedule with one shared $\tau\sim\mathrm{Uniform}[0,1]$ and no mask. This branch lets the same network denoise a full chunk from pure noise during the episode-start warm start. Architecturally, the only required change is to make AdaLN per-position: one $(\gamma_{p},\beta_{p})$ pair per chunk position instead of one shared pair.

One NFE Per Call

At episode start, the policy performs standard-flow inference to initialize the whole buffer and re-noises it to $\boldsymbol{\tau}^{\star,d}$ for the current delay. From then on, every call applies one Euler substep:

$$\mathbf{x}_{p}\leftarrow\mathbf{x}_{p}+\Delta\tau_{p}\,v_{\theta}(\mathbf{x},\boldsymbol{\tau},\mathbf{o})_{p},\qquad \tau_{p}\leftarrow\tau_{p}+\Delta\tau_{p}$$

The per-position advances $\Delta\tau_{p}$ shift the schedule right by $d$ slots. Positions $[d,2d)$ reach $\tau=1$ and are emitted, the remaining buffer rotates forward, and $d$ fresh-noise slots are appended. When the measured $d$ changes, the pattern of advances adapts and pulls the buffer toward the new schedule over a few calls.

Code Correspondence

The official repository implements latency adaptation in deployment/apps/run_policy.py. The following _effective_d() logic is the client-side counterpart of the paper's delay parameter $d$: continuous mode uses the measured rolling query latency, while pipelined mode takes the larger of measured latency and chunk length so the server does not slide past what the client consumes.

def _latency_ticks():
    if not _query_times:
        return max(1, int(args.chunk_len or 1))
    mean_ms = float(np.mean(_query_times))
    return max(1, int(np.round(mean_ms / (period * 1000.0))))

def _effective_d():
    measured = _latency_ticks()
    is_rtc = bool(args.force_nonstreaming) or (args.ckpt_type == "rtc")
    if is_rtc or args.query_mode == "continuous":
        return measured
    return max(measured, max(1, int(args.chunk_len or 1)))

The asynchronous vision mechanism lives in deployment/mindex/policy/groot_client.py. update_vlm_cache() pushes fresh image, state, and language to the VLM server in a background thread. get_action_chunk_cached() then sends a state-only fast-path request to the DiT server, skipping image transforms and the VLM forward:

obs = {"state": self._build_state_dict(state_history)}
options = self._build_options(
    num_inference_timesteps=num_inference_timesteps,
    slide_steps=slide_steps,
    inpaint_actions=inpaint_actions)
data = {"observation": obs, "options": options,
        "t_state_capture": t_state_capture}
response = self._call("get_action_chunk_cached", data=data)

The training fork exposes the matching flags: --streaming-schedule-mode pir2, --streaming-chunk-size-max 5, --image-delay-max 5, and a 0.2 standard-flow warm-up branch through --streaming-constant-weight. These correspond to the staircase branch, the learned delay embedding, and the warm-start branch in the paper.

flowchart LR
  subgraph SLOW["Asynchronous slow channel"]
    IMG["Image + language"] --> VLM["GR00T VLM backbone"]
    VLM --> CACHE["Cached VL features
+ delay embedding e(d_vlm)"]
  end
  subgraph FAST["Fast 25 Hz action loop"]
    PROP["Fresh proprioception"] --> DIT["DiT action head
one NFE per call"]
    CACHE --> DIT
    DIT --> STEP["One Euler substep
+ slide d positions"]
    STEP --> EMIT["Emit d clean actions
from [d, 2d)"]
  end
  EMIT --> ROBOT["xArm6 + XHand position control"]
  ROBOT --> PROP

πR² data flow. The slow VLM cache supplies semantic conditioning while fresh proprioception drives one-step denoising and action sliding every control tick.

Experiments

The simulation study uses Leap Cube Reorientation in MuJoCo Playground. Demonstrations come from four PPO experts and total 200 trajectories; all policies are state-based flow policies with chunk length $H=16$. The first experiment sweeps execution horizon $h$ at zero inference delay. Standard flow degrades as $h$ grows, while πR² replans every timestep with one NFE per call and matches standard flow with $h\in\{1,2\}$.

The second experiment evaluates realistic VLA latency. With $d_{0}$ defined as the cost of four denoising steps, end-to-end baselines pay $1.75d_{0}$ per call, πR² without async processing pays $1.0d_{0}$, and πR² with async processing pays $0.25d_{0}$. Results below are taken from Figure 3; each cell averages 100 episodes over three seeds and selects the best epoch per seed.

Method	$d_{0}=1$	$d_{0}=2$	$d_{0}=3$
Naive Async	0.33	0.29	0.22
Train-Time RTC	0.36	0.32	0.19
πR² with async	0.43	0.42	0.45

The margin grows as delay increases. Baselines accumulate effective action delay linearly, while πR² with async processing holds action delay at one tick. Visual delay grows, but success degrades gracefully, supporting the claim that vision provides coarse guidance while up-to-date proprioception drives reactive manipulation.

Figure: simulation results. Left shows the execution-horizon tradeoff and single-step replanning benefit; right shows success under growing effective latency.

The real-world evaluation fine-tunes GR00T-N1.7 on an xArm6 + XHand system at 25 Hz on an RTX A5000. The four tasks, Don't Spill, Tidy Up Book, Insert Box, and Catch Book, require contact-rich and reactive behavior. Each cell uses N=20 trials; SR is whole-task success and Prog is the fraction of completed subgoals.

Figure: the four real-world tasks. Don't Spill requires carrying a bowl without dropping the ball, Tidy Up Book requires extracting and placing a book, Insert Box requires pushing a box upright and inserting it, and Catch Book requires catching a falling book.

Method	Don't Spill SR/Prog	Tidy Up SR/Prog	Insert Box SR/Prog	Catch Book SR
Flow Sync (h=10)	4/20 · 16/80	4/20 · 9/40	11/20 · 56/80	4/20
Flow Naive Async + TE	7/20 · 30/80	7/20 · 15/40	12/20 · 61/80	2/20
Train-Time RTC	9/20 · 45/80	8/20 · 18/40	10/20 · 53/80	5/20
πR²	10/20 · 55/80	12/20 · 24/40	16/20 · 68/80	11/20

πR² outperforms every baseline on every metric. The largest gains occur on tasks that depend on live contact feedback. Synchronous inference freezes the robot at chunk boundaries and creates jitter; naive asynchronous inference with temporal ensembling smooths motion but sacrifices precision; train-time RTC anchors the next chunk to a clean prefix of executed actions but inherits a bias toward the previous motion, which slows failure recovery.

The force trace in Figure 5 explains the quantitative gap. On Tidy Up Book, πR² modulates grip from live fingertip force and stops near roughly 50 N, just enough to reorient the book. Train-Time RTC reacts late, pushes down after contact, and overshoots to roughly 120 N, crushing the book. Similar patterns appear on Catch Book, Insert Box, and Don't Spill: πR² closes on force spikes while RTC arrives too late.

Figure: fingertip force and emitted action over time on Tidy Up Book. πR² stops near 50 N while RTC overshoots to about 120 N.

Limitations

The authors state that πR² does not address latency sources external to the model, including communication delays between an inference server and the robot client. The method can measure such delay and adapt the schedule, but it cannot remove jitter introduced by the link itself.

The second stated limitation is architectural: the base policy structure is kept unchanged. The paper does not add dedicated attention heads or other mechanisms that make proprioceptive tokens especially expressive inside the DiT. The authors argue that such changes could further amplify reactivity and leave them to future work.

From an engineering perspective, the released code also documents an approximation in streaming Path B: the server-side streaming buffer initialization does not exactly reproduce the derived chunk_size=d staircase, so the deployment currently requires manual alignment of chunk length with measured latency. The real-world evaluation is also concentrated on one platform, xArm6 + XHand, with a specific set of dexterous tasks; broader cross-embodiment conclusions need more evidence.

Conclusion and Outlook

πR² redefines the real-time problem for flow-based VLA policies. Instead of making the entire VLA faster, it makes the action loop independent of the slow semantic backbone: fresh proprioception drives one-step denoising, cached VLM features provide coarse semantic conditioning, and a delay-adaptive staircase absorbs the remaining computation and communication time.

The results are strongest for contact-rich and dynamic manipulation, where reactive force feedback is more important than semantic planning. Future work can strengthen proprioceptive attention inside the action head, model external communication delay explicitly, and transfer the same scheduling mechanism to world-action models and other embodiments. The official code is open, and both the training flags and deployment loop expose the paper's design choices directly.

Real-time closed-loop control is not about making the backbone faster; it is about letting the action loop stop waiting for it.