Papers

GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture

Vision-language-action (VLA) models have become a dominant paradigm for generalist embodied agents, demonstrating strong complex and long-horizon task completion in structured settings. Yet it remains an open question whether current VLA systems can benefit from more effective architectural design, scale to substantially larger and more heterogeneous data regimes, and achieve broader generalization across tasks and embodiments. To this end, we present GigaBrain-0.7, an embodied foundation model with substantially improved generalization across diverse robot embodiments. Specifically, GigaBrain-0.7 unifies understanding, prediction, and action through a three-system architecture, scales pretraining to over 37,000 hours of heterogeneous embodied data, and introduces one-stage alignment training that jointly optimizes vision-language understanding and multi-embodiment action generation. Compared with the preceding GigaBrain-0 series and prior state-of-the-art models including $π_{0.5}$, GigaBrain-0.7 achieves substantial improvements in foundation zero-shot capabilities, language-conditioned instruction following, and post-training task success rates. In particular, on our in-house Maker H01 platform and mainstream robot embodiments, GigaBrain-0.7 demonstrates strong task adaptability and completion ability across both home and industrial scenarios. All training code and pretrained model weights will be released.

VLA具身智能世界模型 GigaBrain Team, Angen Ye, Axiang Sun·Aug 16, 2026

LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics

LeVJEPA performs video self-supervised pretraining with a single encoder, a single loss and one fixed hyperparameter (λ=0.02): an invariance loss plus SIGReg regularization provably rule out representation collapse, with no target encoder, predictor, stop-gradient or pixel reconstruction. It uses 5.6–20.8× less training compute than V-JEPA 2, leads by 7.6 points on ImageNet-1K under a FLOP-matched budget, and gets block-causal attention for free — paving the way to streaming perception and autoregressive world models.

视频自监督预训练JEPA表征坍缩Lukas Kuhn, Lucas Maes, Giuseppe Serra·Aug 27, 2026

Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization

Zero-shot cross-task generalization, where a policy must execute manipulation tasks never seen during training, remains a central challenge in robot learning. In large language models, a novel task can be performed simply by specifying it in the context, without any parameter update. This form of in-context learning (ICL) turns generalization into a problem of task specification. To achieve cross-task generalization, we bring this paradigm to robotic manipulation, and argue that the natural task specification for manipulation is a human video: unlike language, it provides rich visual cues about the intended task evolution. We present Zero-WAM, a causal video-action model that executes unseen tasks by following in-context human video guidance. To address the scarcity of task-rich paired human-robot data, we propose an automatic pipeline that converts task-sampled robot trajectories into semantically matched human videos, yielding HumanGen, a dataset of 74.2K human-robot ICL pairs across 8.6K tasks. For model training, we further introduce an in-context future chunk prediction (IFP) objective that suppresses shortcuts learned from seen tasks and forces the policy to draw task information from the video prompt. On seven unseen tasks in RoboTwin 2.0 simulation, Zero-WAM achieves a 47.0% average success rate, an absolute improvement of 29.5 percentage points over the strongest video-action baseline. In real-world evaluations, it follows human video guidance to generalize to unseen task configurations involving multi-object scenes, long-horizon manipulation, and fine-grained insertion.

世界模型上下文学习人类视频示教Jiaming Zhou, Qihang Zhang, Gangwei Xu·Aug 26, 2026

DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation

DECOWAM adapts a frozen FastWAM video-action backbone to legged mobile manipulation via decoupled interfaces — an action-equivalent future bottleneck, adversarial base/arm factorization, and ego-motion-aware video conditioning — cutting Stage-2 trainable parameters 232x while leading real-robot deployment at 58.2% success.

世界模型VLA移动操作Siyuan Ma, Boshi Zhang, Yutian Zhang·Aug 20, 2026

GigaBrain-WBC-0.5: A Behavior World Model for Robust Whole-Body Control with Environment Interaction

The first Behavior World Model (BWM) for humanoid whole-body control. A causal Transformer jointly predicts next action, state, and latent behavior command distribution, enabling the policy network to model how the environment shapes actions. Automatic terrain-annotation pipeline recovers 3D contact geometry from retargeted motion. At deployment, implausible commands are detected and retracted onto learned behaviors. Achieves highest success rate across all four regimes: 81.3% terrain interaction, 83.1% under implausible commands, 99.3% fall recovery. Unitree G1 checkpoint transfers to Maker L01 robot.

人形机器人Humanoid世界模型Ziyang Cheng, Tianshu Tang, Jinxin Lan·Aug 18, 2026

StageWAM: Joint-Embedding Stage Prediction for World-Action Models in Robot Manipulation

Generalist robot policies aim to map multimodal observations and linguistic task instructions to actions across diverse tasks. However, existing methods typically represent the future as a fixed, short video-action chunk. This short-term future captures local scene evolution for action execution, but it does not explicitly describe the stage-level future that specifies how a task should progress from its current stage to the next. We therefore distinguish two complementary futures for robot manipulation: a short-term physical future to capture local scene evolution and a stage-level semantic future to represent task progress. We introduce StageWAM, which augments a Motus-based World Action Model (WAM) with Stage-JEPA, a goal-conditioned Joint-Embedding Predictive Architecture (JEPA) predictor. Given the current observation and task instruction, Stage-JEPA uses a frozen V-JEPA2 encoder to extract the current-state representation and predicts the latent target of the next inferred stage. Across 50 RoboTwin 2.0 tasks in clean and randomized environments, StageWAM achieves 90.25% overall success and reduces the mean number of execution steps in successful rollouts by 5.97% relative to the strongest baseline.

机器人操作世界模型JEPAXiao Liu, Yuguang Yang, Xi Wang·Aug 11, 2026

World Action Models (WAMs): Unifying VLA and World Models

Introduces World Action Models (WAMs) to unify predictive state modeling with action generation, addressing VLA limitations. Proposes taxonomy and analyzes data ecosystem.

具身智能Embodied AIVLASiyin Wang, Junhao Shi, Zhaoyang Fu·May 12, 2026

PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud

Physical AI policies require inference throughout their lifecycle, including model evaluation, cloud reinforcement learning rollout, edge GPU serving, and onboard deployment. Although these settings share the same checkpoint and action semantics, they often rely on separate inference programs. To unify them, we build PhyAI, a Physical AI inference engine with a single runtime that keeps architecture-specific conditioning, solver, cache, and output logic in model adapters while sharing graph execution, kernels, memory management, and parallel services. The same codebase runs vision-language-action (VLA) models and world-action models (WAMs) on single or multiple GPUs across onboard, edge, and cloud deployments. We used the adapter interface to add MiniCPM-Robot on the day of its release. PhyAI achieves 1.40x-4.65x speedups over the official implementations of pi0, pi0.5, GR00T N1.7, and MiniCPM-Robot. On Cosmos3-Nano-Policy-DROID it reduces latency from 2.46 to 1.18 s on eight H20 GPUs (CFG=2, TP=4), a 2.08x speedup. Specialized runtimes remain faster in several configurations, so our goal is one runtime with competitive latency rather than the fastest result in every case. Detailed profiles reveal why different models need different execution policies: on a Hopper-series GPU at batch size one, the pi0.5 action expert accounts for 8.8% of FLOPs but 57.2% of latency; at batch size 32 its share drops to 13.5% and throughput reaches about 100 samples/s. Cosmos3 remains generation-dominated and gains only 14.3% throughput as batch size increases from 1 to 16. We further introduce the control-time Roofline, which distinguishes inference-bound from environment-bound control; the measured pi0.5 points on four LIBERO suites are environment-bound while Cosmos3 stays inference-bound. Code and benchmarks: https://github.com/mingti-org/phyai.

Physical AI推理引擎VLAChenghua Wang, Daliang Xu, Dongqi Cai·Aug 4, 2026

DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priors

DeWorldSG generates spatio-temporally robust 3D semantic scene graphs from RGB-D sequences. It estimates instance-level geometric 3D Gaussian distributions through depth-guided filtering and represents each object as a probabilistic 3D node. It aggregates spatiotemporal evidence across object pairs and refines relations using contextual priors from a world model (V-JEPA 2). Improves triplet recall by 77.4% and predicate recall by 23.2% over prior SoTA.

3D场景图scene graphRGB-DSeok-Young Kim, Abdelrahman Elskhawy, Taewook Ha·Jul 1, 2026

ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation

ω-0 is a latent predictive whole-body world-action model for real-world humanoid concurrent loco-manipulation: given a language instruction, visual observation, and proprioceptive state, it directly predicts controller-compatible whole-body action latents, coupling compact future-observation embedding prediction with diffusion-based action generation. A 40+ hour real-world dataset ω-HOME is collected; a single model outperforms IL, VLA, humanoid, and WAM baselines on 11 household tasks.

人形机器人世界模型移动操作Zhe Li, Zhenzhe Zhang, Yangyang Wei·Aug 6, 2026

PhyLatent: Learning Dynamics-Relevant Representations for JEPA World Models

We propose PhyLatent, a dynamics-relevant training objective for JointEmbedding Predictive Architecture (JEPA) world models. Our key observation is that preventing global latent collapse does not ensure that a representation preserves physical states and action consequences. We identify three failure modes in JEPA world models: physical invariance collapse, physical identifiability collapse, and counterfactual dynamics collapse. PhyLatent addresses them through three training pathways: physical invariance, physical identifiability, and counterfactual dynamics, implemented with physical state grounding, future representation alignment, static visual invariance, counterfactual branch separation, and latent denoising. On OGBench-Cube, PhyLatent reduces the three failure rates from 15.60%, 6.71%, and 8.41% to 7.53%, 0.95%, and 4.62%, respectively, and improves model predictive control (MPC) success from 70.0% to 78.1%. With the same architecture and planner, it further improves success from 81.0% to 98.0% on TwoRooms and remains competitive on Reacher and PushT. These results show that global non-collapse alone is insufficient for learning a reliable JEPA worldmodel state space.

JEPAWorld ModelsMPCXi Zeng, Haojie Ren, Ziying Song·Aug 6, 2026

Robust-WAM: Bridging Generative Pretraining and Semantic Foresight in World-Action Models

Mainstream World-Action Models (WAMs) adapt pretrained video generation models (VGMs) for robot control, transferring their learned dynamics prior for action prediction. These VGMs are typically trained in a variational autoencoder (VAE) latent space. However, the VAE latent space is optimized for pixel reconstruction, which rewards fine appearance detail and leaves the action prediction fragile under visual shifts. Recent works build WAMs in semantic latent space, which are more robust to appearance shifts. However, these models cannot leverage the large-scale VGM pretraining that exists only in VAE space. To overcome this dilemma, we propose Robust-WAM, a general post-training method for video-generation-based WAMs that preserves the VAE-based generative path and adds a lightweight semantic foresight alignment objective on the action stream. This retains the large-scale VGM pretraining while grounding actions in appearance-invariant dynamics that stay reliable under illumination shifts and other visual out-of-distribution conditions. Specifically, we employ learnable query tokens to bring future-scene semantics into the action stream by aligning their output hidden states with the semantic foresight of future ground-truth frames. To establish the temporal correspondence between each query and the future step it describes, we give it the positional encoding of the matching action tokens. Experiments on out-of-distribution generalization simulation benchmarks and a real-robot setup show that our Robust-WAM consistently improves the success rates of multiple WAM baselines without sacrificing in-distribution performance.

世界模型世界-动作模型视频生成Haodong Yan, Junfeng Li, Junjie He·Aug 6, 2026