PAPER DEEP DIVE
LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments
World Action Models (WAMs) have emerged as a powerful paradigm for embodied intelligence, yet the prevailing reliance on pixel-level video generation creates a fundamental bottleneck. Forcing models to reconstruct task-irrelevant visual details dissipates representational capacity and renders policies vulnerable to visual distractors. In this paper, we propose LeapBot-WA, which establishes a novel Predictive-Latent paradigm for WAMs by operationalizing the Joint-Embedding Predictive Architecture (JEPA) as a World-Anchor. Departing from the traditional reliance on visual synthesis, LeapBot-WA shifts the core of world modeling to Predictive Semantic Alignment, extracting abstract physical dynamics directly within a latent foundation space. To bridge the modality gap between non-Gaussian predictive features and diffusion priors, we introduce the Isotropic Semantic Autoencoder (ISAE), which reshapes the anchor's latent space into a diffusion-friendly manifold to prevent off-manifold drift. Furthermore, we design an Asymmetric Mixture-of-Transformers (MoT) architecture. During training, an Anchor Diffusion Transformer acts as a privileged dynamics expert to guide the Action Diffusion Transformer; at inference, this heavy dynamics branch is pruned, enabling zero-overhead execution. LeapBot-WA achieves state-of-the-art performance among predictive models on LIBERO and matches top-tier generative WAMs on RoboTwin 2.0 without requiring large-scale trajectory pre-training. It further demonstrates superior zero-shot robustness to unseen environments and successful real-world transfer, establishing a highly efficient and robust latent-centric paradigm for scalable robotic control. Code: https://github.com/LeapWM/leapbot-wa.
1. Overview: Anchoring World-Action Models via Predictive Latent Alignments
LeapBot-WA, proposed by Pei Liu, Nan Zheng, Lang Zhang et al. in July 2026, establishes a novel Predictive-Latent paradigm for World Action Models (WAMs). Current WAMs rely on pixel-level video generation, forcing models to reconstruct task-irrelevant visual details, dissipating representational capacity and rendering policies vulnerable to visual distractors. LeapBot-WA operationalizes the Joint-Embedding Predictive Architecture (JEPA) as a World-Anchor, shifting the core of world modeling from visual synthesis to Predictive Semantic Alignment—extracting abstract physical dynamics directly within a latent foundation space. To bridge the modality gap between non-Gaussian predictive features and diffusion priors, it introduces the Isotropic Semantic Autoencoder (ISAE), which reshapes the anchor's latent space into a diffusion-friendly manifold to prevent off-manifold drift. An Asymmetric Mixture-of-Transformers (MoT) architecture is designed: during training, an Anchor DiT acts as a privileged dynamics expert guiding the Action DiT; at inference, this heavy branch is pruned for zero-overhead execution. LeapBot-WA achieves state-of-the-art performance among predictive models on LIBERO and matches top-tier generative WAMs on RoboTwin 2.0 without large-scale trajectory pre-training, demonstrating superior zero-shot robustness and real-world transfer.
2. Core Problem: The Representation Bottleneck of Pixel-Level Generation
World Action Models enhance policies with foresight by predicting environment evolution, but mainstream methods rely on pixel-level video generation. This creates a fundamental bottleneck: forcing models to reconstruct task-irrelevant visual details such as lighting and background textures dissipates representational capacity and makes policies fragile to visual distractors. LeapBot-WA asks: can the core of world modeling shift from "rendering the future" to "imagining the abstract physical semantics of the future"—retaining foresight while avoiding the reconstruction burden?
3. Method: Dual-Pathway Predictive-Latent Framework
3.1 Framework Overview
LeapBot-WA maps raw observations to control actions via a dual-pathway architecture: a representation pipeline driven by a Predictive Anchor, and a generative pipeline driven by an Asymmetric MoT. Given multi-view observations $\mathbf{o}$, language instruction $l$, and proprioceptive state $s$, the semantic latent $\mathbf{z}$ and shared context $\mathbf{c}$ are defined as:
$$\mathbf{z} = \mathrm{ISAE}(\Phi_{\text{JEPA}}(\mathbf{o})), \qquad \mathbf{c} = \Psi_{\mathrm{enc}}(l, s) \tag{1}$$where $\Phi_{\text{JEPA}}$ is the Predictive Anchor. LeapBot-WA optimizes a joint diffusion objective coupling the action trajectory $\mathbf{a}$ with the semantic evolution $\mathbf{z}$, ensuring the derived policy is physically grounded by the anchor's predictive priors.
3.2 Diffusion-Friendly Semantic Autoencoding (ISAE)
The fundamental challenge is that predictive anchor features are non-Gaussian while diffusion priors assume Gaussianity. The ISAE reshapes the anchor's latent space into an isotropic manifold, making high-level semantic priors generative-ready for stable action synthesis. Let $\Phi_{\text{JEPA}}(\mathbf{o})$ output semantic tokens $\mathbf{u}$; the ISAE encodes $\mathbf{z} = E_{\text{ISAE}}(\mathbf{u})$ with reconstruction loss:
$$\mathcal{L}_{\text{rec}} = \|\mathbf{u} - D_{\text{ISAE}}(\mathbf{z})\|_2^2 + \lambda_{\text{iso}} \mathcal{L}_{\text{SIGReg}}(\mathbf{z})$$where $\mathcal{L}_{\text{SIGReg}}$ is isotropic regularization preventing dimensional collapse. Ablations show: without SIGReg the latent rank is only 38.1 with gradient norm variance 1.25; with SIGReg rank restores to 92.3, variance drops to 0.18, and multi-step drift falls from 0.45 to 0.21.
3.3 View-Aware Residual Conditioning
The shared-weight ISAE projects multi-view observations into a unified feature space without explicit camera priors, but downstream multi-view dynamics modeling still requires camera identity. To retain multi-view spatial awareness without breaking symmetric encoding, view residuals are introduced directly at the Anchor DiT input. Given a noisy semantic latent $\mathbf{z}^{(v),t}$ at diffusion step $t$:
$$\mathbf{h}^{(v)} = \mathrm{Proj}\!\left(\mathrm{Norm}(\mathbf{z}^{(v),t})\right) + \mathbf{e}^{(v)}_{\mathrm{view}} \tag{8}$$where $\mathbf{e}^{(v)}_{\mathrm{view}}$ is a learned embedding for view $v$. This decomposition lets semantic content, camera identity, and diffusion conditioning play distinct roles in the downstream transformer.
3.4 Intent-Conditioned Dynamics Modeling
The Anchor DiT branch operates exclusively within the ISAE latent space, aiming to anticipate future semantic states of the world. Bypassing the representation bottleneck of pixel-level generation, it focuses on abstract, action-relevant dynamics (e.g., topological changes of a cloth being folded). Crucially, it is formulated as an intent-conditioned world model: explicitly conditioned on shared context $\mathbf{c}$ (language instruction and proprioception) but entirely independent of the target action trajectory $\mathbf{a}$. By deliberately depriving the world model of step-by-step low-level motor commands, semantic representations are prevented from collapsing into trivial control shortcuts. The model is forced to learn a generalized, intent-driven physical prior—imagining the high-level semantic consequences of a task based solely on the robot's current physical configuration.
3.5 Asymmetric Latent Dynamics Distillation
To transfer abstract dynamics from the intent-conditioned world model to the control policy, an asymmetric distillation mechanism jointly denoises semantic states and actions. Semantic and action sequences are independently corrupted with flow-matching noise at a shared diffusion timestep $t$:
$$\mathbf{z}^t = (1-t)\mathbf{z} + t\boldsymbol{\epsilon}_z, \qquad \mathbf{a}^t = (1-t)\mathbf{a} + t\boldsymbol{\epsilon}_a \tag{9}$$The noisy tokens are processed by Anchor DiT and Action DiT within the Asymmetric MoT. The core design is asymmetric masked interaction with concatenated tokens $\mathbf{H}^{(\ell)}=[\mathbf{H}_s^{(\ell)}; \mathbf{H}_a^{(\ell)}]$ (Eq. 10) and joint masked self-attention:
$$\mathrm{Attn}_{\mathrm{asym}}(\mathbf{H}^{(\ell)}) = \mathrm{Softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^\top}{\sqrt{d}} + \mathbf{M}\right)\mathbf{V} \tag{11}$$The asymmetric mask $\mathbf{M}$ blocks the Anchor DiT from attending to action tokens while preserving the Action DiT's attention to semantic tokens. This induces an elegant hierarchical control structure: the Anchor DiT acts as a visionary dreaming the goal-directed future, while the Action DiT distills dynamics from its semantic foresight. At inference, the Anchor DiT is pruned for zero-overhead execution.
4. Experiments: Simulation and Real-World
4.1 Simulation Results
On the RoboTwin 2.0 benchmark, LeapBot-WA achieves a 91.46% average success rate, significantly outperforming pre-trained VLAs like $\pi_{0.5}$ and X-VLA. In randomized settings it reaches 92.34%, setting a new SOTA among WAMs and surpassing models using extensive robot pre-training (e.g., Lingbot-VA). On LIBERO suites, LeapBot-WA achieves the best performance within the predictive WAM category at 97.3% success, significantly exceeding predictive baselines like PALM and VLA-JEPA while remaining highly competitive with more computationally expensive generative models such as Motus and ImageWAM.
| Benchmark | LeapBot-WA | Category | Highlight |
|---|---|---|---|
| RoboTwin 2.0 avg | 91.46% | Predictive WAM | Beats $\pi_{0.5}$, X-VLA |
| RoboTwin 2.0 randomized | 92.34% | WAM SOTA | Beats Lingbot-VA (needs large pre-training) |
| LIBERO mean | 97.3% | Best predictive WAM | Beats PALM, VLA-JEPA |
| LIBERO-Plus zero-shot | 73.1% | Predictive WAM SOTA | Large margin over VLA-JEPA |
4.2 Real-World Results
Deployed on a real UR5 robot, the task requires identifying, picking, and placing various fruits and vegetables into designated containers under different table textures and lighting. Despite the domain gap between simulation and real-world, the agent maintains precise control and exhibits smooth, stable trajectories, confirming that representation quality and the regularized latent space effectively transfer to reliable downstream execution.
4.3 Ablation Studies
Cross-Domain Transfer and Representation Quality: Linear probing shows mixed-domain pre-training consistently reduces MSE—action prediction by 4.6%, dynamics prediction by 37.9%, and proprioception by 28.5%. Impact of Isotropic Regularization on Latent Geometry: Without SIGReg, latent rank is 38.1 with gradient variance 1.25 (representation collapse and instability); with SIGReg, rank restores to 92.3, variance drops to 0.18, multi-step drift from 0.45 to 0.21, interpolation jerk from 5.1 to 1.2, and success rate from 42.5% to 71.3%.
| Latent Geometry Metric | Without SIGReg | With SIGReg | Improvement |
|---|---|---|---|
| Latent Rank | 38.1 | 92.3 | ↑142% |
| Gradient Variance | 1.25 | 0.18 | ↓85.6% |
| Multi-step Drift | 0.45 | 0.21 | ↓53.3% |
| Interpolation Jerk | 5.1 | 1.2 | ↓76.5% |
| Success Rate | 42.5% | 71.3% | ↑67.8% |
5. Limitations
- Foundation model dependency: Performance remains tied to the representational quality of the underlying predictive foundation model; if the foundation under-represents certain domains, LeapBot-WA is limited.
- Fine-grained contact physics: The current latent-centric approach excels in spatial and semantic understanding, but modeling extremely fine-grained contact physics or high-frequency tactile interactions may require further alignment.
- Long-horizon planning unextended: The paradigm has not yet been extended to long-horizon task planning, left as future work.
- Multi-modal priors not integrated: Future work will explore integrating audio-visual or haptic priors into the world-anchor space.
6. Conclusion
LeapBot-WA demonstrates that the core utility of world modeling for robotic manipulation lies in abstract physical anticipation rather than photorealistic rendering. By leveraging a predictive-latent space and the ISAE to bridge representational gaps, it establishes a framework where heavy dynamics branches act as privileged experts during training but are pruned at inference, achieving zero-overhead action generation with superior zero-shot robustness. It achieves predictive WAM SOTA on LIBERO (97.3%), matches top-tier generative WAMs on RoboTwin 2.0 (91.46%) without large-scale pre-training, and reaches 73.1% zero-shot on LIBERO-Plus. The core insight: when a privileged expert that "dreams" future semantics guides the action generator during training and is then pruned at inference, foresight is distilled into the policy—adding no overhead.
flowchart LR
A["Multi-view obs o"] --> B["Predictive Anchor Φ_JEPA
(fine-tuned JEPA)"]
B --> C["Semantic tokens u"]
C --> D["ISAE isotropic autoencoder"]
D --> E["Diffusion-friendly latent z"]
L["Language instr l"] --> F["Shared context c = Ψ_enc(l, s)"]
S["Proprioception s"] --> F
F --> G["Anchor DiT
(intent-conditioned world model)"]
E --> G
G -->|"asymmetric masked attention
distill dynamics"| H["Action DiT"]
E --> H
H --> I["Action trajectory a"]
I --> J["Zero-overhead inference
(Anchor DiT pruned)"]
