Skip to content
RobotWorld
Back to Papers

PAPER DEEP DIVE

世界模型VLA移动操作

DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation

DECOWAM adapts a frozen FastWAM video-action backbone to legged mobile manipulation via decoupled interfaces — an action-equivalent future bottleneck, adversarial base/arm factorization, and ego-motion-aware video conditioning — cutting Stage-2 trainable parameters 232x while leading real-robot deployment at 58.2% success.

Siyuan Ma, Boshi Zhang, Yutian Zhang, Qinglian Wu, Jiaqi Zhai, Dong Wei, Qiaojun YuAugust 20, 202621 min read
中文

Authors: Siyuan Ma, Boshi Zhang, Yutian Zhang, Qinglian Wu, Jiaqi Zhai, Dong Wei, Qiaojun Yu
Affiliations: Tsinghua University · Shanghai AI Laboratory · Harbin Institute of Technology · Hangzhou Yunshenchu Technology (DEEP Robotics)
Paper: arXiv:2608.20114 (2026-08-20, cs.AI) · Code: not released (no repository link in the paper) · Dataset: ARMDOG (planned release)

One-Sentence Summary

DECOWAM treats legged mobile manipulation as a decoupled world-action modeling problem: base control, arm manipulation, and camera ego-motion enter a frozen FastWAM video-action backbone through explicit, semantically aligned interfaces, cutting Stage-2 trainable parameters 232x while improving every video and action metric and leading real-robot deployment.

DECOWAM architecture, training, and deployment

Fig. 1: DECOWAM architecture. A frozen WAN video expert and an ActionDiT action expert receive language and proprioceptive context; decoupled conditional interfaces specialize the backbone for the quadruped-arm embodiment.

1. Background and Motivation

A mobile manipulator must predict how locomotion and arm motion jointly alter future observations and control. Vision-language-action (VLA) models such as RT-2, OpenVLA, π0, Octo, RDT-2, and X-VLA map language and pixels to actions with transformer policies pretrained on large heterogeneous corpora, and adapted π0.5 and X-VLA runs serve as action-only references in this paper. Their standard action heads, however, do not explicitly model future RGB rollout, nor do they separate base-induced camera motion from arm-induced scene change.

World-model approaches generate future observations and use them either as learned simulators or as structural priors on the policy. Recent video-based world-action models — UniSim, UVA, X-WAM, FastWAM, and Motus — combine visual prediction with action modeling. But these systems were developed largely for fixed-base platforms: they do not explicitly distinguish camera ego-motion from base and arm actions. DECOWAM deliberately does not pursue general simulator scaling; it studies embodiment-specific adaptation, i.e., how to keep a pretrained video prior useful while separating ego-motion, base action, arm action, and future-equivalent control information on a legged-manipulation dataset.

Why is legged-arm modeling harder than fixed-base manipulation? The paper names three structural obstacles. First, dynamic viewpoint: the on-board camera moves with the base, hand-eye geometry varies continuously, and image streams contain a mixture of ego-motion and scene motion — a mobile-manipulation world model needs an explicit route for representing camera motion rather than treating all pixel displacement as scene dynamics. Second, multi-rate action coupling: arm joint trajectories require high-rate control (~15-30 Hz) while base velocity commands are typically issued at ~3-5 Hz; concatenating both into one uniformly sampled action chunk asks a single representation to cover navigation-scale velocity and manipulation-scale joint corrections at once. Third, hierarchical intent: real tasks interleave "where to go" with "how to act," and a monolithic latent struggles to represent navigation-scale decisions and manipulation-scale corrections simultaneously.

The central idea of DECOWAM is a decoupled modeling paradigm: a world-action model should represent where the base moves, how the arm acts, and how camera ego-motion changes future pixels as explicit factors. It is implemented on top of FastWAM, a Wan-2.2-based world-action backbone with a paired ActionDiT branch. Four mechanisms realize the paradigm: an action-equivalent future bottleneck distilled from privileged future latents; staged frozen adaptation that keeps the base FastWAM prior fixed and learns only residual robot-specific pathways; a base/arm dual latent with gradient-reversal factorization; and a base-velocity token as the explicit ego-motion interface for the video branch.

Method development is only half of the story: legged mobile manipulation needs data in which visual change, base ego-motion, arm motion, and language intent are synchronized rather than recorded as separate logs. The authors therefore build ARMDOG, a real-robot resource for a wheeled quadruped with a 6-DoF arm. Its contribution is the embodiment-complete model interface: each converted episode aligns a 15 Hz RGB stream, a T×14 whole-body state/action tensor with explicit base and arm channels, instruction text, and a precomputed language embedding.

2. Preliminaries: The FastWAM World–Action Backbone

DECOWAM builds on FastWAM, a world–action backbone that couples a Wan-2.2 video diffusion backbone with an ActionDiT branch predicting chunked actions under the same continuous-flow training interface. In the ARMDOG configuration, FastWAM consumes the current RGB frame, a precomputed language context, and a 14-D proprioceptive state token, then jointly produces future RGB frames and a 48-step whole-body action chunk. The checkpoint evaluated in this paper has 6725.44M total parameters, of which 6020.75M are trainable during full fine-tuning.

Two design consequences follow from this backbone choice. First, both output branches are trained with conditional flow matching, so the action expert and the video expert share one optimization interface — any conditioning mechanism added to one branch has an exact analogue in the other. Second, because the backbone is a large pretrained video model, indiscriminate fine-tuning on a small legged-manipulation corpus risks overwriting the visual prior that makes future-frame prediction feasible in the first place. DECOWAM's staged frozen adaptation is a direct response to this risk: keep the joint video/action factorization intact, and add four mechanisms that decouple the adaptation problem in parameter space, future-information usage, action factors, and ego-motion conditioning. The paper is explicit that it does not pursue general simulator scaling; the question is how to keep a pretrained video prior useful while separating ego-motion, base action, arm action, and future-equivalent control information on a single embodiment.

3. Method in Detail

3.1 Problem Formulation

Let ℓ denote a language instruction, x0 the current RGB observation, and s0 ∈ ℝ^14 the current whole-body state. DECOWAM learns the conditional joint model:

$$p_{\theta}\bigl(x_{1:T},\mathbf{a}_{1:K}\mid x_{0},s_{0},\ell\bigr)$$

where x_{1:T} is a future video and a_{1:K} ∈ ℝ^{K×14} is an action chunk, with T = 8 and K = 48. Each action has the semantic decomposition:

$$\mathbf{a}_{k}=\bigl[\mathbf{a}^{\mathrm{arm}}_{k},a^{\mathrm{grip}}_{k},\mathbf{a}^{\mathrm{base}}_{k},\mathbf{a}^{\mathrm{pad}}_{k}\bigr],\qquad\mathbf{a}^{\mathrm{base}}_{k}\in\mathbb{R}^{3}$$

The arm and gripper occupy channels [0:7], base velocity occupies [7:10], and loader padding occupies [10:14]. Base velocity is both a control target and a source of camera ego-motion — the key tension DECOWAM resolves by separating arm control, base control, and visual ego-motion instead of encoding them in one undifferentiated context.

3.2 Staged Parameter-Efficient Adaptation

Training separates domain alignment from structural adaptation. In Stage 1, all FastWAM parameters Θ are adapted to ARMDOG for 50k steps:

$$\Theta^{(1)}=\arg\min_{\Theta}\mathbb{E}_{\mathcal{D}}\bigl[\mathcal{L}_{\mathrm{video}}(\Theta)+\mathcal{L}_{\mathrm{action}}(\Theta)\bigr]$$

This stage aligns the video prior, action expert, and proprioceptive interface with moving-camera observations and the quadruped-arm action space. Stage 2 freezes Θ^(1) and optimizes only the four new parameter groups Φ = {φ_adp, φ_q, φ_ba, φ_ego}:

$$\Phi^{\star}=\arg\min_{\Phi}\mathcal{L}\bigl(\Theta^{(1)},\Phi\bigr)$$

These groups represent residual adapters, the action-equivalent future bottleneck, base/arm factorization, and ego-motion conditioning. The restriction reduces the Stage-2 trainable footprint from 6020.75M to 25.95M parameters. The frozen WAN backbone is adapted after each block through a bottleneck residual:

$$h_{l}^{+}=h_{l}+\alpha_{l}W_{\mathrm{up}}^{(l)}\sigma\bigl(W_{\mathrm{down}}^{(l)}\mathrm{LN}(h_{l})\bigr)$$

where $W_{\mathrm{down}}$ projects to a 128-D bottleneck, $W_{\mathrm{up}}$ restores the hidden dimension, and $\sigma$ is SiLU. The residual branch learns a compact robot-specific correction while preserving the pretrained video prior.

Interface 1: Action-Equivalent Future Bottleneck

Future frames carry action-equivalent outcome information — where the object ends up, whether the gripper closed, whether the base reached its target — that the current frame alone cannot provide. Training can observe the future; deployment cannot. DECOWAM resolves this with privileged-teacher → causal-student distillation.

Current and future frames are encoded to WAN-VAE latents and summarized by mean and standard deviation:

$$e_{0}=\psi_{\mathrm{vae}}(x_{0}),\qquad e_{1:T}=\psi_{\mathrm{vae}}(x_{1:T})$$ $$c=\rho(e_{0}),\qquad f=\rho(e_{1:T}),\qquad \rho(e)=\bigl[\operatorname{mean}(e),\ \operatorname{std}(e)\bigr]$$

Teacher and student embeddings are then:

$$z_{t}=q_{t}([c,f,s_{0}]),\qquad z_{s}=q_{s}([c,s_{0}]),\qquad z_{t},z_{s}\in\mathbb{R}^{d_{q}}$$

The teacher sees the future summary $f$ (privileged); the student sees only the current summary $c$ and robot state $s_0$ (causal). Only the student embedding conditions the causal action expert through a residual bias:

$$\tilde{u}^{a}=u^{a}+\eta_{q}B_{q}z_{s}$$

with $\eta_{q}\in[0,1]$ scaling the residual. The bottleneck loss combines action reconstruction, teacher–student distillation, and geometry preservation:

$$\mathcal{L}_{q}=\lambda_{\mathrm{act}}^{q}\mathcal{L}_{\mathrm{rec}}^{q}+\lambda_{\mathrm{dist}}^{q}\lVert z_{s}-\operatorname{sg}(z_{t})\rVert_{2}^{2}+\lambda_{\mathrm{geom}}^{q}\mathcal{L}_{\mathrm{geom}}$$

The reconstruction term requires both teacher and student to recover the 48-step action chunk:

$$\mathcal{L}_{\mathrm{rec}}^{q}=\lVert r_{s}(z_{s})-\mathbf{a}_{1:K}\rVert_{2}^{2}+\lVert r_{t}(z_{t})-\mathbf{a}_{1:K}\rVert_{2}^{2}$$

The geometry term keeps the structure of the privileged latent space aligned with the structure of action space — trajectories with similar actions should stay close in embedding space. For a batch of $B$ samples, pairwise distances are robustly scaled by batch medians:

$$d_{z}^{ij}=\frac{\lVert z_{t}^{i}-z_{t}^{j}\rVert_{2}}{\tau_{z}},\qquad d_{a}^{ij}=\frac{\lVert \mathbf{a}^{i}_{1:K}-\mathbf{a}^{j}_{1:K}\rVert_{2}}{\tau_{a}}$$ $$\mathcal{L}_{\mathrm{geom}}=\bigl(B(B-1)\bigr)^{-1}\sum_{i\neq j}\operatorname{SL1}\bigl(d_{z}^{ij},d_{a}^{ij}\bigr)$$

Distillation transfers this structure to the deployable student. At inference the teacher $q_t$ and all auxiliary heads are removed — the model conditions strictly on $(x_0, s_0, \ell)$ with no future leakage.

Future-information bottleneck: privileged teacher vs causal student

Figure 2: The future-information bottleneck. A privileged teacher observes current and future visual summaries; a causal student observes only the current summary and robot state. Only the student survives deployment.

Interface 2: Base–Arm Factorization

The 14-D action vector mixes navigation-scale information (base velocity: low-rate, moves the robot) with manipulation-scale information (arm joints: high-rate, moves the end effector). DECOWAM maps the pooled action context into two 16-D factors:

$$z_{\mathrm{base}}=b_{\phi}(u^{a}),\qquad z_{\mathrm{arm}}=m_{\phi}(u^{a}),\qquad z_{\mathrm{base}},z_{\mathrm{arm}}\in\mathbb{R}^{16}$$

Their concatenation adds a second residual conditioning layer to the action expert:

$$\bar{u}^{a}=\tilde{u}^{a}+\eta_{\mathrm{ba}}B_{\mathrm{ba}}[z_{\mathrm{base}},z_{\mathrm{arm}}]$$

To make $z_{\mathrm{base}}$ genuinely encode only base information, the paper uses adversarial disentanglement with a gradient-reversal layer (GRL). Writing $\mathbf{a}^{b}_{1:K}=\mathbf{a}_{1:K,\,7:10}$ (base channels) and $\mathbf{a}^{m}_{1:K}=\mathbf{a}_{1:K,\,0:7}$ (arm+gripper channels), the disentanglement loss is:

$$\mathcal{L}_{\mathrm{disent}}=\lVert g_{b}(z_{\mathrm{base}})-\mathbf{a}^{b}_{1:K}\rVert_{2}^{2}+\lVert g_{m}(z_{\mathrm{arm}})-\mathbf{a}^{m}_{1:K}\rVert_{2}^{2}+\lVert \tilde{g}_{b}(\operatorname{GRL}(z_{\mathrm{arm}}))-\mathbf{a}^{b}_{1:K}\rVert_{2}^{2}+\lVert \tilde{g}_{m}(\operatorname{GRL}(z_{\mathrm{base}}))-\mathbf{a}^{m}_{1:K}\rVert_{2}^{2}$$

The first two terms are direct heads: each factor must reconstruct its assigned control quantity. The last two are cross heads routed through GRL, whose sign-flipped gradients push each encoder to discard the other factor's information. Each latent retains its assigned control factor and nothing else.

Interface 3: Ego-Motion-Aware Video Conditioning

Apparent image motion seen by a body-mounted camera is a mixture of three sources: scene dynamics, manipulator motion, and base-induced viewpoint change. DECOWAM exposes the third component explicitly — the normalized current base velocity extracted from proprioception:

$$v_{0}=\Pi_{\mathrm{base}}(s_{0})=(v_{x},v_{y},\omega_{z})\in\mathbb{R}^{3}$$

Every video token receives the same projected ego-motion condition:

$$\tilde{h}_{i}^{v}=h_{i}^{v}+\beta B_{v}v_{0},\qquad i=1,\ldots,N_{v}$$

This token does not impose geometric warping; it supplies an explicit explanatory variable for camera-frame motion. Base velocity thus plays a dual role: a control target in the action branch (channels $[7:10]$ of Eq. (2)) and a visual condition in the video branch (Eq. (16)) — the decoupling philosophy made concrete at the interface level.

Training Objective and Deployment

Both experts use conditional flow matching. For a target $y\in\{\mathbf{a}_{1:K},e_{1:T}\}$, noise $\epsilon\sim\mathcal{N}(0,I)$, and time $\tau\sim\mathcal{U}(0,1)$:

$$y_{\tau}=(1-\tau)\epsilon+\tau y,\qquad v^{\star}(y_{\tau},\tau)=y-\epsilon$$ $$\mathcal{L}_{\mathrm{FM}}(F_{\theta};y,c)=\mathbb{E}_{\tau,\epsilon}\bigl[\lVert F_{\theta}(y_{\tau},\tau,c)-v^{\star}(y_{\tau},\tau)\rVert_{2}^{2}\bigr]$$

This loss is instantiated as $\mathcal{L}_{\mathrm{action}}$ with context $\bar{u}^{a}$ and as $\mathcal{L}_{\mathrm{video}}$ with context $\tilde{h}^{v}$. The full Stage-2 objective is:

$$\mathcal{L}=\lambda_{v}\mathcal{L}_{\mathrm{video}}+\lambda_{a}\mathcal{L}_{\mathrm{action}}+\gamma_{q}\lambda_{q}\mathcal{L}_{q}+\gamma_{\mathrm{ba}}\lambda_{\mathrm{ba}}\mathcal{L}_{\mathrm{disent}}$$

The paper sets $\lambda_{v}=\lambda_{a}=1.0$, $\lambda_{q}=0.2$, $\lambda_{\mathrm{ba}}=0.1$, with $\gamma_q$, $\gamma_{\mathrm{ba}}$, $\eta_q$, $\eta_{\mathrm{ba}}$ held at one throughout Stage 2. At inference, the teacher and auxiliary heads are removed; the model computes $z_s$, $(z_{\mathrm{base}},z_{\mathrm{arm}})$, and $v_0$ from current inputs and samples both flows using only $(x_0,s_0,\ell)$ — video and actions are generated simultaneously.

flowchart TD
    A["x0 current RGB
+ s0 state + language"] --> B["WAN-VAE encode
e0 = vae(x0)"] B --> C["summary c = mean,std of e0"] C --> D["causal student qs
zs = qs([c, s0])"] D --> E["residual bias on action expert
u~a = ua + eta_q Bq zs"] A --> F["action context pool"] F --> G["base factor z_base 16-D"] F --> H["arm factor z_arm 16-D"] G --> I["concat and bias
u~bar a = u~a + eta_ba Bba [z_base, z_arm]"] H --> I A --> J["base velocity v0 from s0"] J --> K["ego-motion token
h~v = hv + beta Bv v0"] I --> L["ActionDiT flow matching
48-step 14-D action chunk"] K --> M["WAN video flow matching
8 future frames 384x320"] L --> N["deployment output"] M --> N

4. The ARMDOG Dataset: An Embodiment-Complete Interface

The method is only half the story. Legged mobile manipulation needs data where visual change, base ego-motion, arm motion, and language intent are synchronized rather than recorded as separate logs. The authors therefore build ARMDOG, a real-robot resource on a wheeled quadruped platform (16 leg joints + 6-DoF arm + 1-DoF gripper) that synchronizes moving-camera RGB-D, proprioception, IMU, base state, whole-body commands, and language instructions.

A FastWAM-compatible conversion aligns raw HDF5 streams to 15 Hz, storing each episode as:

$$e_{i}=(V_{i},Q_{i},\ell_{i},\phi(\ell_{i}))$$

i.e., RGB video, a structured $T_i\times 14$ state/action tensor, instruction text, and a precomputed language embedding. The full quality-filtered corpus contains 1,487 episodes, 343,550 frames, about 321.3 minutes at 15 Hz. Task mix: Bottle Pick&Place 56%, Place Block 39%, Object Pick&Place 4%, Climb Slope 1% — object-centric mobile manipulation with a small locomotion slice. Downstream subsets are reported separately from the full corpus: frozen Stage-2 uses 214 episodes across 26 tasks, while all replay results use the fixed Box-val slice (23 episodes, 8 tasks, 4,323 frames).

ARMDOG dataset composition and model interface

Figure 3: ARMDOG task distribution, corpus scale, and model interface. The 14-D action tensor explicitly separates 6-D arm, 1-D gripper, 3-D base-velocity, and 4-D loader-padding channels.

5. Experimental Results

5.1 Replay Protocol and Module Ablation

Open-loop replay uses a fixed 23-episode box_val slice (8 task folders covering box_move/box_soft/box_stay variants). All models are compared under identical 16 replay batches with shared normalization and evaluator code. The internal ablation shows every removed structured module degrades all metrics:

VariantF-MSE ↓PSNR ↑SSIM ↑A-MSE ↓A-MAE ↓
Full frozen decoupled9.35e-431.3789.9241e-18.09e-54.310e-3
w/o future bottleneck1.02e-331.2289.9170e-18.40e-54.316e-3
w/o base-velocity1.01e-331.2219.9178e-18.80e-54.342e-3
adapter-only (w/o decoupled)9.80e-431.1059.9138e-19.60e-55.135e-3

Relative to the adapter-only control, the full model reduces frame MSE by 4.6%, action MSE by 15.7%, and action MAE by 16.1%. Removing either the future bottleneck or base-velocity conditioning hurts both output branches — the future-equivalent representation and explicit ego-motion signal improve the shared video–action model beyond residual adaptation alone.

The evaluation metrics span four complementary axes. Video quality uses frame MSE, PSNR, global SSIM, and LPIPS — jointly covering pixel fidelity, structural agreement, and perceptual similarity. Action prediction uses MSE, MAE, and the mean Euclidean error of each normalized 14-D action vector (A-L2), evaluated over the complete arm–gripper–base interface that the deployed system actually uses. All reported results are computed over the same 16 replay batches with shared normalization, inputs, and evaluator code, which is what makes the cross-model comparisons in the following tables meaningful. The FastWAM baseline family exposes checkpoints at 40k, 50k, and 80k Stage-1 steps to show sensitivity to the stopping point; DECOWAM starts from the 50k checkpoint and freezes all FastWAM parameters during Stage 2, with a trainable path consisting of 128-D WAN adapters, a 64-D action-equivalent future bottleneck, 16-D base and arm latents, and base-velocity conditioning.

5.2 Action Replay vs VLA References

ModelFamilyRGBA-MSE ↓A-MAE ↓A-L2 ↓
π0.5VLAno1.79e-43.60e-32.83e-2
X-VLAVLAno2.11e-51.82e-31.00e-2
GR00TVLAno4.18e-32.51e-21.83e-1
DECOWAMWAM8f5.38e-54.01e-32.24e-2

Dedicated action policies are indeed stronger: X-VLA attains the lowest error on all three metrics. Yet DECOWAM ranks second on A-MSE and A-L2, improving them by 69.9% and 21.1% over π0.5; its A-MAE is within 11.1% of π0.5 and 84.0% lower than GR00T. Crucially, it simultaneously outputs a 48-step whole-body action and eight 384×320 future RGB frames — the VLA references stop at the action chunk.

5.3 Comparison with World–Action Model References

ModelTrain. paramsRGBF-MSE ↓PSNR ↑SSIM ↑LPIPS ↓A-MSE ↓A-MAE ↓A-L2 ↓
DECOWAM25.95M8f 384p8.77e-431.669.93e-12.95e-25.38e-54.01e-32.24e-2
FastWAM6020.75M8f 384p1.03e-331.449.92e-13.03e-26.87e-54.08e-32.35e-2
Motus5894.81M8f 384p5.19e-323.419.57e-11.01e-15.05e-49.97e-35.99e-2
Cosmos 2.58f 384p4.28e-214.386.63e-12.71e-1
X-WAM5037.75M4f 160p2.62e-326.579.81e-13.50e-26.31e-41.11e-22.48e-2
UVA261.62M4f 128p1.79e-219.328.40e-12.00e-19.76e-41.31e-29.81e-2

Among WAM references, DECOWAM ranks first on every reported video and action metric. Under the matched eight-frame 384×320 interface it lowers FastWAM frame/action MSE by 15.03%/21.71%; relative to Motus by 83.1%/89.3%; relative to the shorter, lower-resolution X-WAM output by 66.6%/91.5%. Notably, DECOWAM trains only 25.95M parameters while FastWAM/Motus/X-WAM all train over 5000M — frozen adaptation works better.

Qualitative WAM rollout comparison

Figure 4: Qualitative comparison on one fixed ARMDOG replay sample. DECOWAM better preserves workspace geometry and object layout while competing rollouts accumulate blur or viewpoint drift.

The qualitative comparison in Figure 4 shows what these numbers look like on a single fixed replay sample: ground-truth future RGB montages sit alongside predictions produced through each model's native interface, with red dashed lines marking camera-view boundaries. DECOWAM preserves workspace geometry and object layout across the eight predicted frames, while several competing rollouts accumulate blur or drift in viewpoint. Viewpoint drift is the symptom most directly tied to this paper's thesis: a world model that cannot account for base-induced camera motion attributes the resulting pixel flow to the scene itself, and its rollouts wander.

5.4 Real-Robot Deployment: 79 Closed-Loop Trials

ModelMean time (s) ↓Approach ↑Grasp ↑Transport ↑Placement ↑Success ↑
GR00T5773.4%13.9%11.4%8.9%8.9%
π0.55089.9%62.0%53.2%49.4%49.4%
FastWAM6591.1%63.3%59.5%57.0%57.0%
X-WAM8277.2%26.6%19.0%15.2%15.2%
DECOWAM4992.4%69.6%67.1%58.2%58.2%
ModelDocking ↑Coordination ↑Base-displacement robustness ↑Autonomous recovery ↑
GR00T70.9%16.5%1.3%11.4%
π0.587.3%36.7%11.4%25.3%
FastWAM83.5%34.2%12.7%27.8%
X-WAM69.6%27.8%5.1%12.7%
DECOWAM87.3%44.3%30.4%32.9%

DECOWAM completes 46/79 tasks (58.2%) with a mean completion time of 49 s — 16 s faster than FastWAM and 33 s faster than X-WAM, whose success rates are 57.0% and 15.2%. It posts the highest approach and transport rates and carries 96.4% of successful grasps into transport. The standout figure in the robustness table is base-displacement robustness at 30.4%: the fraction of trials that recover after the base is physically pushed — nearly triple π0.5. This is the direct payoff of modeling base motion as an explicit factor.

The stage decomposition deserves a closer reading. Because every stage denominator is the full 79 attempts, the columns form a strict survival chain — approach, grasp, transport, placement — and the gaps between consecutive columns localize where each model loses trials. GR00T loses 59.5 percentage points between approach and grasp: the platform reaches the workspace but cannot reliably acquire the object, consistent with an action-only VLA lacking any anticipatory model of contact geometry. X-WAM collapses even earlier, losing over 50 percentage points at the approach stage before the grasp phase begins. DECOWAM's chain is the flattest of all five methods: 92.4 → 69.6 → 67.1 → 58.2. Its largest single drop, 22.8 points at the grasp, is still smaller than the corresponding drop of every competitor, and the transport-to-placement gap of only 8.9 points means that once an object is moving, DECOWAM almost always finishes. Carrying 96.4% of successful grasps into transport is the clearest behavioral signature of a model that predicts, rather than reacts to, the consequences of base motion.

Real-robot whole-body coordinated-motion trials

Figure 5: Representative whole-body coordinated-motion trials on the quadruped–arm platform.

5.5 Parameter Efficiency and Deployment Cost

ModelTotal (M)Trainable (M)Latency (ms) ↓F-MSE ↓A-MSE ↓
FastWAM6725.446020.751196.61.032e-36.87e-5
DECOWAM6751.3825.951333.28.77e-45.38e-5

Stage-2 trainable parameters shrink roughly 232-fold (6020.75M → 25.95M) while both frame and action MSE improve, at the cost of only 11.4% added evaluator latency. One distinction matters: freezing reduces optimization cost, not deployed network size — the full 6.75B-parameter network still runs forward, and total size actually grows slightly (6751.38M vs 6725.44M) from adapter and conditioning overhead.

This distinction between training-side and inference-side cost is what makes the deployment profile an interesting operating point rather than a free lunch. The 232-fold reduction in trainable parameters means that a lab with modest GPU budget can specialize a 6.75B world-action model on a few hundred episodes without catastrophic forgetting of the video prior, without optimizer-state memory blowup, and with a short, reproducible Stage-2 run. What it does not buy is edge deployability: every inference still pays for the full WAN backbone, which is why the evaluator latency rises from 1196.6 ms to 1333.2 ms even though the learned delta is tiny. The paper's framing is precise about this trade-off — it reports a "causal video–action interface with explicit embodiment factors, future-frame prediction, and parameter-efficient robot specialization" as its operating point, and leaves model distillation for real-time control as future work.

5.6 Related work in one view

The paper positions itself between three literatures. Fixed-base VLAs — RT-1/RT-2, OpenVLA, π0, Octo, RDT-2, X-VLA, PaLM-E, SayCan, Gato, RoboCat, plus task-conditioned policies such as BC-Z, PerAct, VIMA, and Diffusion Policy — map language and pixels to actions through large pretrained transformers, and the replay experiments include adapted π0.5 and X-VLA runs as action-only references; their standard action heads neither predict future RGB rollouts nor separate base-induced camera motion from arm-induced scene change. World models for control — World Models (Ha and Schmidhuber), DreamerV3, Genie, UniSim, UVA, X-WAM, FastWAM, and Motus — generate future observations either as learned simulators or as structural priors on the policy; DECOWAM deliberately does not pursue simulator scaling, asking instead how a pretrained video prior can stay useful while ego-motion, base action, arm action, and future-equivalent control information are separated on a single embodiment. Legged locomotion contributes the control substrate: reinforcement-learning controllers that output base-velocity commands, typically teleoperated or loosely coupled with a separate manipulator policy — exactly the hierarchical "where to go versus how to act" split that the decoupled interfaces are designed to represent.

Why does freezing beat fine-tuning here?

The ablation suite and the deployment table together suggest a concrete mechanism. Stage 1 already aligns the video prior, the action expert, and the proprioceptive interface with moving-camera quadruped observations over 50k steps — by that point the backbone's failure modes are dominated not by missing embodiment knowledge but by the three entanglements identified in the motivation: ego-motion masquerading as scene dynamics, navigation-scale and manipulation-scale signals sharing one latent, and future-outcome information unavailable at inference. Each of DECOWAM's four mechanisms targets exactly one of these residual failure modes with a low-dimensional pathway (128-D adapters, a 64-D bottleneck, two 16-D factors, a 3-D velocity token). Unfreezing the 6B backbone at this stage would let the optimizer re-spend capacity on problems the prior already solves — with only 214 Stage-2 episodes, the expected outcome is overfitting, not improvement. The empirical pattern matches: every metric improves when the backbone is frozen, and every structural pathway contributes individually (removing any one degrades both branches). The 25.95M trainable footprint is small enough to learn the embodiment-specific corrections and too small to damage the prior.

6. Limitations

1. Limited data scale and task diversity (author-stated). 95% of the ARMDOG corpus concentrates on two tabletop placement tasks (Bottle Pick&Place 56% + Place Block 39%), with only 1% locomotion-focused episodes. The decoupled interfaces are validated on structured approach–grasp–transport–placement flows; whether the gains persist on long-horizon navigation, dynamic obstacle avoidance, or multi-object rearrangement remains untested.

2. The deployed network remains huge. Frozen adaptation makes training cheap (25.95M trainable), but deployment must run the full 6.75B-parameter network forward at 1333 ms per evaluation (11.4% more than FastWAM). For a legged platform needing high-rate closed-loop correction, this inference cost can bottleneck real-time operation — the parameter efficiency is on the training side, not the inference side.

3. No public code. As of this writing the paper provides no GitHub repository, so implementation details of GRL disentanglement and teacher–student distillation (exact $\lambda_{\mathrm{dist}}^q$, $\lambda_{\mathrm{geom}}^q$ values and schedules) cannot be independently verified.

7. Conclusion and Outlook

DECOWAM's answer is crisp: a world model for legged mobile manipulation does not need a bigger video backbone — it needs the right interfaces. Pull base control, arm manipulation, and camera ego-motion out of an undifferentiated context and inject them through four small, explicit structures (residual adapters, future bottleneck, base/arm factors, ego-motion token), and a frozen 6B video prior outperforms full fine-tuning: 15% lower frame error, 22% lower action error, 58.2% real-robot success with leading whole-body coordination and perturbation tolerance.

The broader lesson: the bottleneck for embodied foundation models is shifting from pretraining scale to the structural design of embodiment adaptation. Once video priors are strong enough, the ceiling on real-robot performance is set by how cheaply and correctly the control semantics of a specific body — which part is navigation, which is manipulation, which is the camera itself moving — are injected. The frozen-backbone-plus-decoupled-interface paradigm is likely to generalize to dual-arm, wheeled, and multi-robot embodied adaptation.

8. Golden Quotes

"A world model for mobile manipulation doesn't lack a bigger video backbone — it lacks a route to tell camera ego-motion, base locomotion, and arm manipulation apart."

"Freeze six billion parameters and teach 26 million to wear this body — it works better than fine-tuning everything."

Related Papers

GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture

GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture

Vision-language-action (VLA) models have become a dominant paradigm for generalist embodied agents, demonstrating strong complex and long-horizon task completion in structured settings. Yet it remains an open question whether current VLA systems can benefit from more effective architectural design, scale to substantially larger and more heterogeneous data regimes, and achieve broader generalization across tasks and embodiments. To this end, we present GigaBrain-0.7, an embodied foundation model with substantially improved generalization across diverse robot embodiments. Specifically, GigaBrain-0.7 unifies understanding, prediction, and action through a three-system architecture, scales pretraining to over 37,000 hours of heterogeneous embodied data, and introduces one-stage alignment training that jointly optimizes vision-language understanding and multi-embodiment action generation. Compared with the preceding GigaBrain-0 series and prior state-of-the-art models including $π_{0.5}$, GigaBrain-0.7 achieves substantial improvements in foundation zero-shot capabilities, language-conditioned instruction following, and post-training task success rates. In particular, on our in-house Maker H01 platform and mainstream robot embodiments, GigaBrain-0.7 demonstrates strong task adaptability and completion ability across both home and industrial scenarios. All training code and pretrained model weights will be released.

VLA具身智能世界模型Aug 16, 2026
LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics

LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics

LeVJEPA performs video self-supervised pretraining with a single encoder, a single loss and one fixed hyperparameter (λ=0.02): an invariance loss plus SIGReg regularization provably rule out representation collapse, with no target encoder, predictor, stop-gradient or pixel reconstruction. It uses 5.6–20.8× less training compute than V-JEPA 2, leads by 7.6 points on ImageNet-1K under a FLOP-matched budget, and gets block-causal attention for free — paving the way to streaming perception and autoregressive world models.

视频自监督预训练JEPA表征坍缩Aug 27, 2026
Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization

Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization

Zero-shot cross-task generalization, where a policy must execute manipulation tasks never seen during training, remains a central challenge in robot learning. In large language models, a novel task can be performed simply by specifying it in the context, without any parameter update. This form of in-context learning (ICL) turns generalization into a problem of task specification. To achieve cross-task generalization, we bring this paradigm to robotic manipulation, and argue that the natural task specification for manipulation is a human video: unlike language, it provides rich visual cues about the intended task evolution. We present Zero-WAM, a causal video-action model that executes unseen tasks by following in-context human video guidance. To address the scarcity of task-rich paired human-robot data, we propose an automatic pipeline that converts task-sampled robot trajectories into semantically matched human videos, yielding HumanGen, a dataset of 74.2K human-robot ICL pairs across 8.6K tasks. For model training, we further introduce an in-context future chunk prediction (IFP) objective that suppresses shortcuts learned from seen tasks and forces the policy to draw task information from the video prompt. On seven unseen tasks in RoboTwin 2.0 simulation, Zero-WAM achieves a 47.0% average success rate, an absolute improvement of 29.5 percentage points over the strongest video-action baseline. In real-world evaluations, it follows human video guidance to generalize to unseen task configurations involving multi-object scenes, long-horizon manipulation, and fine-grained insertion.

世界模型上下文学习人类视频示教Aug 26, 2026
GigaBrain-WBC-0.5: A Behavior World Model for Robust Whole-Body Control with Environment Interaction

GigaBrain-WBC-0.5: A Behavior World Model for Robust Whole-Body Control with Environment Interaction

The first Behavior World Model (BWM) for humanoid whole-body control. A causal Transformer jointly predicts next action, state, and latent behavior command distribution, enabling the policy network to model how the environment shapes actions. Automatic terrain-annotation pipeline recovers 3D contact geometry from retargeted motion. At deployment, implausible commands are detected and retracted onto learned behaviors. Achieves highest success rate across all four regimes: 81.3% terrain interaction, 83.1% under implausible commands, 99.3% fall recovery. Unitree G1 checkpoint transfers to Maker L01 robot.

人形机器人Humanoid世界模型Aug 18, 2026