Physical AI

Videos (51)

47:24

Meta Reality Labs open-sources Project SuperDex, a contact-first dexterous manipulation simulator

@IntuitMachine · 19 views · 2026-08-28

灵巧操作Dexterous Manipulation仿真

0:50

Lightwheel Releases EgoSuite-Open100K: Open Egocentric Dataset for Physical AI

@LightwheelAI · 10 views · 2026-08-26

第一人称数据具身智能Physical AI

0:44

DexRobot open-sources SO101 Pick Cube SFT checkpoint and DM0.5 LoRA SFT workflow

@DexRobotics · 9 views · 2026-07-28

DexRobotSO101DM0.5

1:14

SPD: VR Simulation Pre-training for Bimanual Dexterity

@yokoshaan · 10 views · 2026-08-23

SPD灵巧操作Dexterous Manipulation

1:06

3D HAMSTER: 3D End-Effector Trajectory from RGB-D

@robotsdigest · 2 views · 2026-08-22

3D HAMSTERVLA操作

0:36

Inside SuperMap: Persistent 3D Object Identities

@robotsdigest · 6 views · 2026-08-16

SLAMSuperMapSuperOdometry

3:52

SuperMap Turns SLAM into Persistent 4D Spatial Memory

@robotsdigest · 5 views · 2026-08-16

SLAMSuperMap空间记忆

1:00

Magenta centaur robot combines dual arms and a wheeled quadruped

@ZappyZappy7 · 11 views · 2026-08-15

半人马机器人双臂轮式四足

1:32

NVIDIA SimFoundry turns real scenes into simulation worlds

@yukez · 7 views · 2026-07-02

SimFoundryNVIDIA仿真

0:37

DeWorldSG: Probabilistic 3D Gaussian Scene Graph Generation (ECCV 2026)

@tokufxug · 5 views · 2026-08-14

3D场景图Scene Graph3DGS

1:02

DJI and XGRIDS air-ground 3DGS map fusion

@DuckbillStudio · 2 views · 2026-08-12

3DGS地图融合DJI

0:59

OpenAI sim-trained robot hand solves Rubiks cube

@GoDrift_ai · 5 views · 2026-08-11

灵巧手sim-to-real模拟训练

0:34

DJI M4E and PortalCam air-ground 3DGS map fusion

@DuckbillStudio · 2 views · 2026-08-07

3DGS无人机DJI Matrice 4E

0:30

piR2: real-time flow policies on contact

@robotsdigest · 4 views · 2026-08-05

流策略πR²实时控制

0:11

PolyLayout: multi-room 3D layout estimation

@ai_bites · 1 views · 2026-08-06

3D布局室内导航感知

0:12

Axis Robotics partners with Boost Robotics

@axisrobotics · 3 views · 2026-08-06

人形机器人仿真Boost Robotics

0:58

Niantic Builds City Digital Twin for Physical AI

@NianticSpatial · 1 views · 2026-08-04

数字孪生仿真Niantic

3:21

GEN-1 Embodied Foundation Model: GPT-3 Moment for Robotics

@robotsdigest · 3 views · 2026-08-04

具身AI基础模型VLA

1:08

UnrealTwin: 36M Splat 3D Snow Simulation on WebGPU

@tokufxug · 5 views · 2026-08-03

3D渲染WebGPU3DGS

1:00

SuperMap: Spatio-Temporal SLAM for Visual-Language Navigation

@ShiboZhaoSLAM · 4 views · 2026-07-14

SLAMSuperMap视觉语言导航

0:36

Neural Harmonic Textures: Real-Time Novel View Synthesis

@Arcanous98 · 18 views · 2026-04-09

新视角合成novel view synthesis3DGS

0:22

REMIND Re-ID Tracker for Indoor Navigation

@rsasaki0109 · 2 views · 2026-07-30

重识别re-identification视觉跟踪

0:37

PRISM-VO: Scale-Aware Visual Odometry via Photometric Plenoptic Bundle Adjustment

@Almorgand · 3 views · 2026-07-29

视觉里程计visual odometry光场相机

0:09

mimic-video Beats pi0 VLAs on LIBERO

@cybernetic_lab · 3 views · 2026-07-29

VLAmimic-videoLIBERO

0:13

Monocular Camera Autonomous Driving Perception

@tokufxug · 3 views · 2026-07-30

自动驾驶autonomous driving单目感知

0:20

ImmersiVERSE: Physics-Based LiDAR Simulation on UE + Isaac Sim

@tokufxug · 1 views · 2026-07-28

仿真SimulationLiDAR

0:19

Xynova Humanoid Robot Learns to Write by Hand

@techniahqrobot · 3 views · 2026-07-27

人形机器人手写操作

0:18

Open-set object detection recognize unseen objects

@ctorobotics · 2 views · 2026-07-26

感知Perception开放集

0:15

Walden Robotics raises 300M at 1.1B valuation

@LeoKharon · 6 views · 2026-07-26

Walden Robotics融资Funding

0:16

Wi-Fi through-wall human sensing open-source system

@0x0SojalSec · 7 views · 2026-07-26

感知PerceptionWiFi

0:34

VLK: Vision-Language-Kinematics for Humanoid Loco-Manipulation

@LeoKharon · 8 views · 2026-07-25

人形HumanoidVLA

0:56

Dexterous Hand Origami vs Japanese Master

@RoboPapers · 3 views · 2026-06-01

灵巧手Dexterous HandManipulation

6:36

Scaling Behavior Foundation Model for Humanoid Robots

@weishuaizeng · 2 views · 2026-05-17

人形HumanoidBFM

1:59

Software Replaces Expensive Robot Sensor Arrays

@EugBass · 2 views · 2026-07-25

Perception感知软件

1:08

Hyundai Partners NVIDIA on Robot Reference Platform

@clankrmedia · 3 views · 2026-07-25

NVIDIAHyundaiPlatform

1:02

Scaling Grasping via Egocentric Human Data

@chris_j_paxton · 5 views · 2026-07-25

抓取GraspingManipulation

1:10

SLAM3R: Real-time 3D Scene from Single Video

@LearnWithSubhan · 8 views · 2026-07-25

SLAM3D重建开源

0:35

Tool pick-and-place: why Physical AI is harder than imagined

@HoaTranRom · 2 views · 2026-07-24

操控manipulation物理AI

0:31

Sharpa MoDE-VLA Dual Dexterous Hands Peel Apple Autonomously

@Olivier__OG · 3 views · 2026-07-19

灵巧手DexterousHandMoDE-VLA

5:04

Xiaomi Xiaomi-Robotics-1 VLA Scaling Laws

@GranzenTon · 4 views · 2026-07-20

VLA缩放定律Xiaomi

0:29

NVIDIA ArtiFixer Repairs Incomplete Gaussian Splats

@justinryanio · 2 views · 2026-07-20

3DGS高斯泼溅空间智能

1:21

FlexionAI: RGB-Based RL Policy Training in Simulation

@FlexionAI · 3 views · 2026-07-20

强化学习RLRGB

0:09

Industrial Robotics: Operating Systems Matter More Than Hardware

@IlirAliu_ · 6 views · 2026-07-20

工业机器人机器人操作系统软件生态

1:47

T800 robot fights in China — robots being blind is the big issue

@cixliv · 2 views · 2026-07-16

HumanoidTwitterVLA

0:13

ClothTransformer — high-quality cloth simulation dataset (493K frames)

@XingangP · 2 views · 2026-07-17

Twitter布料仿真

0:14

Robot autonomously traverses a room while building a map

@nikhilkr · 2 views · 2026-07-19

TwitterVLA感知

4:06

An incredible diversity of robots running in the real world, demonstrating real applications. Including end-to-end models

@serious0nhc · 10 views · 2026-07-19

HumanoidTwitterVLA

0:13

✨🇨🇳At the 2026 World Artificial Intelligence Conference in Shanghai, ...

@XueJia24682 · 4 views · 2026-07-19

HumanoidTwitter人形

Papers (54)

WilLaGS: Latent-Conditional 3D Appearance Fields for Robust Gaussian Splatting In-the-Wild

3D Gaussian Splatting (3DGS) delivers real-time and high-fidelity rendering but remains challenged by unconstrained in-the-wild scenes, where drastic appearance variations and transient objects violate multi-view consistency. Existing methods are fundamentally limited by independent and discrete embeddings that struggle to capture continuous environmental changes or model spatially-varying local illumination. To address these limitations, we propose \textbf{WilLaGS}, a unified framework for robust 3D scene reconstruction and generative appearance synthesis under unconstrained settings. Specifically, we introduce a generative appearance model where a $β$-VAE learns a structured and continuous manifold of global appearance. Conditioned on the latent code, we construct a 3D neural appearance field that generates dynamic Tri-Plane features to encode spatially-varying local illumination effects. Furthermore, to suppress transient artifacts, we present a self-supervised perceptual masking mechanism that leverages a Teacher-Student (EMA) architecture to derive a stable scene consensus, robustly identifying inconsistent regions via perceptual discrepancies. Extensive experiments on multiple datasets demonstrate that \textbf{WilLaGS} achieves state-of-the-art performance in reconstruction quality and novel view appearance synthesis, while maintaining real-time rendering efficiency.

Yuhao Bai, Qianqiu Tan, Lilong Chen· 2026-08-28T11:55:53

3DGSGaussian Splatting新视角合成野外重建外观建模

LAC: Linear and Angular Compliance for Humanoid Whole-body Control

LAC is a general humanoid whole-body controller that simultaneously realizes commanded linear and angular compliance for wrenches applied to the upper body. Sampled force and couple events imposed on contact frames from human interaction data synthesize a large-scale compliant-response dataset; teacher-student RL then trains a single policy to track compliant motions under external wrenches, with sim and real experiments showing monotonic stiffness modulation and teleoperated loco-manipulation.

Yang Liu, Zhongkai Gu, Wei Zhu· 2026-08-26T06:07:28

人形机器人全身控制柔顺控制导纳控制强化学习

SOLO: Stable Omni-terrain Long-Horizon Perceptive Humanoid Locomotion

Humans traverse complex terrain over long distances without losing balance, whereas perceptive humanoid policies become fragile as perception and control errors accumulate. We present SOLO, a unified framework addressing two compounding causes of this long-horizon fragility: dense terrain reconstruction smooths action-critical details, and pointwise imitation lacks temporal credit assignment. Its Query Reconstructor (QR) uses Fourier-encoded cell queries to retrieve spatially specific evidence from depth-proprioception tokens, preserving sharp terrain boundaries. Trajectory-Aware MSE (TA-MSE) Distillation adds next-state teacher-student disagreement to the PPO reward, enabling Generalized Advantage Estimation to propagate future disagreement penalties to preceding actions. In simulation, QR reduces height-map L1 error by factors of 3.3-4.0, while TA-MSE surpasses PPO and MSE+PPO in curriculum progression. On stress-test terrains, SOLO achieves 97.5% mean traversal success and 96% stepping-stone success, versus 75.0-75.6% and 0-3% for dense-reconstructor variants. Deployed zero-shot with only a chest-mounted depth camera and proprioception, SOLO completes a continuous 1.5-km outdoor route and an indoor mixed-terrain course. Project page: https://sunpihai-up.github.io/solo/

Pihai Sun, Gang Han, Jingkai Sun· 2026-08-27T03:49:55

humanoid locomotionperceptive locomotionteacher-studentpolicy distillationterrain reconstruction

DELTA: Deformable Elevation-Based Local Terrain Attention Encoder for Sparse-Terrain Quadrupedal Locomotion

Stable quadrupedal locomotion on sparse terrain requires selecting state-relevant terrain evidence for precise foot placement. Model-based foothold planners provide precise foothold selection but rely heavily on explicit model assumptions. Recent attention-based map encoding (AME) studies show that end-to-end reinforcement learning (RL) can learn implicit foothold guidance. However, the computational cost of dense AME encoding grows with map resolution, limiting its scalability to fine-grained sparse terrain. We propose DELTA, a Deformable Elevation-Based Local Terrain Attention encoder. DELTA predicts state-conditioned sampling locations, forms terrain evidence tokens from adaptive local elevation patches, and attends only to a fixed-size token set. With fixed sampling and patch settings, DELTA's encoder cost is independent of map resolution. Experiments show that DELTA achieves final traversal performance comparable to AME at the standard resolution while improving learning efficiency. This fixed encoder cost enables the use of higher-resolution terrain maps, improving traversal on fine-grained sparse terrain. DELTA also demonstrates strong generalization to unseen mixed evaluation courses composed of continuous and discrete terrain elements. Beyond simulation, DELTA demonstrates successful sim-to-real transfer on RAIBO2. Analysis of the learned sampling offsets and attention weights shows that DELTA samples steppable regions and attends to terrain evidence relevant to future touchdowns without foothold labels or attention supervision.

Sanghyun Park, Moonkyu Jung, Jemin Hwangbo· 2026-08-22T00:00:00

四足机器人四足运动强化学习地形感知注意力机制

GigaBrain-WBC-0.5: A Behavior World Model for Robust Whole-Body Control with Environment Interaction

The first Behavior World Model (BWM) for humanoid whole-body control. A causal Transformer jointly predicts next action, state, and latent behavior command distribution, enabling the policy network to model how the environment shapes actions. Automatic terrain-annotation pipeline recovers 3D contact geometry from retargeted motion. At deployment, implausible commands are detected and retracted onto learned behaviors. Achieves highest success rate across all four regimes: 81.3% terrain interaction, 83.1% under implausible commands, 99.3% fall recovery. Unitree G1 checkpoint transfers to Maker L01 robot.

Ziyang Cheng, Tianshu Tang, Jinxin Lan· 2026-08-18T00:00:00

人形机器人Humanoid世界模型World Model全身控制

Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data

Human athletes demonstrate versatile and highly-dynamic tennis skills to successfully conduct competitive rallies with a high-speed tennis ball. However, reproducing such behaviors on humanoid robots is difficult, partially due to the lack of perfect humanoid action data or human kinematic motion data in tennis scenarios as reference. In this work, we propose LATENT, a system that Learns Athletic humanoid TEnnis skills from imperfect human motioN daTa. The imperfect human motion data consist only of motion fragments that capture the primitive skills used when playing tennis rather than precise and complete human-tennis motion sequences from real-world tennis matches, thereby significantly reducing the difficulty of data collection. Our key insight is that, despite being imperfect, such quasi-realistic data still provide priors about human primitive skills in tennis scenarios. With further correction and composition, we learn a humanoid policy that can consistently strike incoming balls under a wide range of conditions and return them to target locations, while preserving natural motion styles. We also propose a series of designs for robust sim-to-real transfer and deploy our policy on the Unitree G1 humanoid robot. Our method achieves surprising results in the real world and can stably sustain multi-shot rallies with human players. Project page: https://zzk273.github.io/LATENT/

Zhikai Zhang, Haofei Lu, Yunrui Lian· 2026-03-13T06:02:30

LATENT人形机器人网球动作学习运动技能

Pre-training Visual Dexterity in Simulation

Large-scale pre-training has made robot policy fine-tuning increasingly data-efficient, but this progress has largely been driven by datasets and embodiments built around simple parallel-jaw grippers. Dexterous, multi-fingered hands remain comparatively data-starved because real teleoperation is costly to scale, while human hand video is off-embodiment and requires lossy pose estimation and retargeting. We introduce Simulation Pre-training for Dexterity (SPD), a pre-training framework for dexterous manipulation that uses data entirely collected in simulation. In SPD, humans manipulate virtual objects inside a VR headset, enabling on-embodiment trajectories and robot-free collection. With the help of five operators, we collect 75 hours of multi-task dexterous manipulation over one week, and use it to pre-train a causal transformer on a sequence modeling objective. We study the benefits of simulation pre-training on real-world tasks by fine-tuning on 1-2 hours of physical demonstrations on a 56-DoF bimanual dexterous setup. We find that our approach outperforms training behavior cloning policies from scratch, showing that simulation teleoperation is a viable pre-training source for real-world dexterous manipulation. We perform ablation studies, measuring the benefits of history conditioning and short action chunks for reactive control.

Sarthak Kamat, Adam Rashid, Satvik Sharma· 2026-08-16T20:21:33

灵巧操作灵巧手预训练sim-to-realVR遥操作

DL-SLAM: Enabling High-Fidelity Gaussian Splatting SLAM in Dynamic Environments based on Dual-Level Probability

Recent advances in 3D Gaussian Splatting (3DGS) have enabled significant progress in dense dynamic Simultaneous Localization And Mapping (SLAM). Prevailing methods typically discard predefined dynamic objects, ignoring that transiently static objects offer valuable geometric constraints for pose estimation. A recent work attempts to leverage this potential by employing per-pixel uncertainty maps to quantify the magnitude of motion. While this approach enables transiently static objects to enhance pose estimation, it erroneously integrates these objects into the static map, resulting in persistent artifacts. Moreover, its reliance on purely geometric information leads to ambiguous object boundaries in the uncertainty maps. To overcome these limitations, we present DL-SLAM, a monocular Gaussian Splatting SLAM system built upon a novel dual-level probabilistic framework. Our method computes dynamic probability maps by combining semantic and geometric information. These pixel-level probabilities are lifted to 3D and aggregated to derive an object-level dynamic probability for each instance. Object-level probability enables the categorical pruning of dynamic Gaussians, resulting in an artifact-free static map. The static map, in turn, provides a geometrically consistent guidance to refine the pixel-wise probabilities, enhancing their reliability. Experimental results demonstrate that DL-SLAM outperforms existing approaches, improving tracking accuracy by up to 13\% while generating high-fidelity semantic maps.

Ziheng Xu, Qingfeng Li, Xuefeng Liu· 2026-07-02T08:18:23

动态环境Dynamic EnvironmentsSLAM3D Gaussian Splatting3DGS

Learning Agile Navigation in Crowded Environments for Quadruped Robots

Navigating dynamic and crowded environments presents significant challenges for quadruped robots due to severe sensor occlusion and unpredictable human motion. Existing approaches face a trade-off: model-based methods, such as Velocity Obstacles (VO), theoretically guarantee safety but rely on accurate obstacle motion estimates that often fail in dense crowds, while end-to-end learning methods offer robustness but lack motion prediction capability of obstacles, leading to collisions or conservative behaviors. To solve this, we propose VOP-Nav, a novel navigation system that combines the geometric safety of VO with the agile adaptability of end-to-end learning. Using only local onboard observations, our system avoids explicit obstacle detection and tracking pipelines. The VOP-Net processes multi-frame LiDAR data to implicitly encode dynamic constraints and predict a safe velocity region derived from Velocity Obstacle theory. Importantly, the VO predictions serve a dual role: they are used as input to the navigation policy during inference and as a reward signal during training to encourage safe motion. Evaluations in Isaac Gym demonstrate that VOP-Nav achieves higher success rates than all baselines while balancing locomotion speed and collision avoidance. Real-world deployment on a Unitree Go2 quadruped robot further validates the system's robustness and efficiency in complex indoor and outdoor dynamic environments.

Shuyu Wu, Zeyu Liu, Tianbao Zhang· 2026-07-16T14:16:19

四足机器人quadruped导航navigation拥挤环境

HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning

HITTER couples a model-based planner (ball trajectory prediction, strike position/velocity/timing) with an RL whole-body controller trained on human forehand/backhand motion references, enabling a Unitree G1 to sustain rallies of up to 106 consecutive shots against a human opponent.

Zhi Su, Bike Zhang, Nima Rahmanian· 2025-08-28T17:49:12

人形Humanoid乒乓球Table Tennis全身控制

VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes

VLK synthesizes paired vision-language-kinematics supervision inside 3DGS-reconstructed real scenes: it generates navigation and object-interaction trajectories with privileged scene info, renders egocentric views after the fact, and produces 48,000 paired trajectories to train a policy predicting Unitree G1 whole-body motion, enabling sim-to-real perception-based humanoid loco-manipulation.

Yen-Jen Wang, Jiaman Li, Sirui Chen· 2026-06-29T17:59:55

humanoid人形机器人loco-manipulation移动操作synthetic data

PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud

Physical AI policies require inference throughout their lifecycle, including model evaluation, cloud reinforcement learning rollout, edge GPU serving, and onboard deployment. Although these settings share the same checkpoint and action semantics, they often rely on separate inference programs. To unify them, we build PhyAI, a Physical AI inference engine with a single runtime that keeps architecture-specific conditioning, solver, cache, and output logic in model adapters while sharing graph execution, kernels, memory management, and parallel services. The same codebase runs vision-language-action (VLA) models and world-action models (WAMs) on single or multiple GPUs across onboard, edge, and cloud deployments. We used the adapter interface to add MiniCPM-Robot on the day of its release. PhyAI achieves 1.40x-4.65x speedups over the official implementations of pi0, pi0.5, GR00T N1.7, and MiniCPM-Robot. On Cosmos3-Nano-Policy-DROID it reduces latency from 2.46 to 1.18 s on eight H20 GPUs (CFG=2, TP=4), a 2.08x speedup. Specialized runtimes remain faster in several configurations, so our goal is one runtime with competitive latency rather than the fastest result in every case. Detailed profiles reveal why different models need different execution policies: on a Hopper-series GPU at batch size one, the pi0.5 action expert accounts for 8.8% of FLOPs but 57.2% of latency; at batch size 32 its share drops to 13.5% and throughput reaches about 100 samples/s. Cosmos3 remains generation-dominated and gains only 14.3% throughput as batch size increases from 1 to 16. We further introduce the control-time Roofline, which distinguishes inference-bound from environment-bound control; the measured pi0.5 points on four LIBERO suites are environment-bound while Cosmos3 stays inference-bound. Code and benchmarks: https://github.com/mingti-org/phyai.

Chenghua Wang, Daliang Xu, Dongqi Cai· 2026-08-04T13:53:48

Physical AI推理引擎VLA世界模型边缘部署

TRACE: Ergodic Trajectory Optimization for Active Scene Reconstruction

Existing active reconstruction systems with Gaussian-splatting maps select observations greedily, optimizing a single next-best-view (NBV) at each step and connecting the chosen views by short-horizon path planning. This greedy decoupling disregards the global structure of scene information, producing inefficient trajectories that waste sensing capacity in transit between selected views. In this work, we study active reconstruction as an ergodic coverage problem: the time-averaged spatial statistics of the sensor trajectory should match a target information distribution induced by the current map. Our approach derives this target distribution online from uncertainty and visibility, and calculates ergodic trajectories via a kernel-ergodic horizon planner with gradient flow and footprint depletion, closing the loop between mapping and trajectory optimization. We thoroughly evaluate TRACE on the Replica dataset against the Next-Best-View (NBV) baselines, improving PSNR by 1.5 dB. Code: https://github.com/spikelab-jhu/trace-active-reconstruction.

Ziyue Zheng, Linli Shi, Bingkun He· 2026-08-03T14:30:20

主动感知轨迹优化遍历覆盖3DGS场景重建

StreamSplat: Streaming Feed-Forward 3D Gaussian Splatting

Feed-forward 3D Gaussian Splatting enables efficient novel-view synthesis without per-scene optimization, but most existing methods assume a fixed set of context views and process them jointly. This limits their applicability to online scenarios where calibrated views arrive sequentially and the scene must be updated causally. We present \emph{StreamSplat}, a streaming feed-forward 3DGS framework that incrementally maintains a persistent geometry-grounded scene state and decodes it into renderable 3D Gaussians after each input chunk. StreamSplat centers on a \textbf{Voxel-Aligned Causal Cache (VACC)}, which stores historical 3D tokens in a memory-bounded voxel structure so that memory grows with explored scene geometry rather than stream length. To better reuse history during causal prediction, we introduce \textbf{History-Projected Depth Anchoring (HPDA)} to project cached geometry as depth guidance for current cost-volume estimation, and \textbf{Cache-Guided Feature Injection (CGFI)} to inject cached latent evidence into Gaussian-token regression. Experiments on DL3DV, RealEstate10K, and ScanNet show that StreamSplat remains competitive with state-of-the-art feed-forward 3DGS methods under sparse causal inputs, despite not using future views or full-scene context. More importantly, it scales to long input streams with 256, 512, and 1024 views where fixed-view baselines run out of memory, yielding sustained improvements in novel-view synthesis quality as more observations arrive. The code will be made publicly available upon acceptance.

Changhao Song, Yuxuan Wang, Qibiao Li· 2026-08-03T03:48:20

Twitter3DGS三维重建流式推理新视角合成

PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball

A perception-aware CBF-RL framework that couples control-barrier safety with onboard sensing for whole-body humanoid dodgeball. The deployed policy sees the ball only as segmentation-masked depth from a head-mounted camera, with CBF guidance representing clearance to every body link. Deployed zero-shot on Unitree G1, achieving 95% success rate on real-world throws.

Lizhi Yang, Junheng Li, Aaron D. Ames· 2026-07-30T17:59:35

人形机器人CBFRLUnitree G1安全控制

Learning Humanoid Standing-up Control across Diverse Postures

Standing-up control is crucial for humanoid robots, with the potential for integration into current locomotion and loco-manipulation systems, such as fall recovery. Existing approaches are either limited to simulations that overlook hardware constraints or rely on predefined ground-specific motion trajectories, failing to enable standing up across postures in real-world scenes. To bridge this gap, we present HoST (Humanoid Standing-up Control), a reinforcement learning framework that learns standing-up control from scratch, enabling robust sim-to-real transfer across diverse postures. HoST effectively learns posture-adaptive motions by leveraging a multi-critic architecture and curriculum-based training on diverse simulated terrains. To ensure successful real-world deployment, we constrain the motion with smoothness regularization and implicit motion speed bound to alleviate oscillatory and violent motions on physical hardware, respectively. After simulation-based training, the learned control policies are directly deployed on the Unitree G1 humanoid robot. Our experimental results demonstrate that the controllers achieve smooth, stable, and robust standing-up motions across a wide range of laboratory and outdoor environments. Videos and code are available at https://taohuang13.github.io/humanoid-standingup.github.io/.

Tao Huang, Junli Ren, Huayi Wang· 2025-02-12T13:10:09

人形机器人强化学习起立控制sim-to-realUnitree-G1

A Framework for Deploying Learning-based Quadruped Loco-Manipulation

Quadruped mobile manipulators offer strong potential for agile loco-manipulation but remain difficult to control and transfer reliably from simulation to reality. Reinforcement learning (RL) shows promise for whole-body control, yet most frameworks are proprietary and hard to reproduce on real hardware. We present an open pipeline for training, benchmarking, and deploying RL-based controllers on the Unitree B1 quadruped with a Z1 arm. The framework unifies sim-to-sim and sim-to-real transfer through ROS, re-implementing a policy trained in Isaac Gym, extending it to MuJoCo via a hardware abstraction layer, and deploying the same controller on physical hardware. Sim-to-sim experiments expose discrepancies between Isaac Gym and MuJoCo contact models that influence policy behavior, while real-world teleoperated object-picking trials show that coordinated whole-body control extends reach and improves manipulation over floating-base baselines. The pipeline provides a transparent, reproducible foundation for developing and analyzing RL-based loco-manipulation controllers and will be released open source to support future research.

Yadong Liu, Jianwei Liu, He Liang· 2025-12-22T01:19:26

四足操作强化学习sim-to-realROS开源框架

Learning Terrain-Aware Whole-Body Control for Perceptive Legged Loco-Manipulation

Legged manipulators integrate exceptional terrain adaptability along with mobile manipulation capabilities, which make them highly promising for deployment in human-centric environments. By coordinating the control of both legs and arms, a whole-body controller can significantly expand the operational workspace of legged manipulators. However, many existing whole-body controllers primarily depend on proprioception and do not incorporate the critical exteroception required for effective terrain topology perception. This limitation can hinder their ability to adapt to varying environmental conditions and navigate complex terrains effectively. In this paper, we introduce TA-WBC, a terrain-aware whole-body control framework for legged manipulators, which features a novel RL-based unified policy tailored to whole-body loco-manipulation tasks in various terrains. Specifically, we employ a hybrid exteroception encoder to extract terrain features, providing an essential basis for the robot to proactively adapt posture and footholds. Furthermore, to facilitate stable cross-terrain loco-manipulation, we propose a novel end-effector sampling method based on the foot contact plane, decoupling manipulation target from base fluctuations. Moreover, a dual-policy distillation module is introduced to integrate expansive whole-body motion with terrain adaptability without catastrophic forgetting. The simulation and real-world experiments validate the robustness of our proposed controller, which leads to a larger reachable space, less tracking error, and reduced unexpected stumbles. This unified policy highlights the promising capabilities of legged manipulators in performing loco-manipulation tasks across complex terrains.

Sikai Guo, Yudong Zhong, Guoyang Zhao· 2026-05-29T14:22:10

腿式操作地形感知全身控制感知强化学习

Autonomous Grasping On Quadruped Robot With Task Level Interaction

Quadruped robots are increasingly used in various applications due to their high mobility and ability to operate in diverse terrains. However, most available quadruped robots are primarily focused on mobility without object manipulation capabilities. Equipping a quadruped robot with a robotic arm and gripper introduces a challenge in manual control, especially in remote scenarios that require complex commands. This research aims to develop an autonomous grasping system on a quadruped robot using a task-level interaction approach. The system includes hardware integration of a robotic arm and gripper onto the quadruped robot's body, a layered control system designed using ROS, and a web-based interface for human-robot interaction. The robot is capable of autonomously performing tasks such as navigation, object detection, and grasping using GraspNet. Testing was conducted through real-world scenarios to evaluate navigation, object selection and grasping, and user experience. The results show that the robot can perform tasks accurately and consistently, achieving a grasping success rate of 75 % from 12 trials. Therefore, the system demonstrates significant potential in enhancing the capabilities of quadruped robots as service robots in real-world environments.

Muhtadin, Mochammad Hilmi Rusydiansyah, Mauridhi Hery Purnomo· 2025-11-30T19:54:35

四足机器人自主抓取任务交互移动操作感知

$N_0$-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens

We present $N_0$-VTLA, a vision-tactile-language-action (VTLA) foundation model capable of (1) fine-grained contact-rich manipulation with tactile perception and tactile-feedback control, and (2) offline policy improvement from stored deployment data. Building on current vision-based backbones, we propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement. During pre-training, the policy learns broad contact priors from NeoData, our large-scale visuo-tactile robot dataset; to our knowledge, $N_0$-VTLA is the first VTLA model pretrained on tactile data at scale. During post-training, we augment the policy with a predictive tactile pathway that distills the contact patterns learned at scale into the fine motion adjustments required by downstream tactile-centric manipulation. For offline policy improvement, we introduce ALTER, an advantage-conditioned offline reinforcement learning method that converts relative progress and trajectory-event comparisons into binary advantage labels for policy training on a fixed deployment corpus, further improving task-specific learning on contact-rich skills such as deformable object manipulation. Across contact-rich benchmarks, $N_0$-VTLA outperforms strong baselines by wide margins: it wins all nine real-robot NeoReal tasks and reaches 63.8% mean success on a twenty-task simulation suite, against 44.0% for the strongest baseline. $N_0$-VTLA policies trained with ALTER reach 75-95% success on three long-horizon real-robot tasks. These results lay a foundation for versatile tactile-driven manipulation policies.

NeoteAI Team, Fudan TEAI Team· 2026-07-26T17:58:47

PaperSimulation仿真

Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping

Operating constrained dynamical systems requires controllers to efficiently solve complex tasks while enforcing recursive feasibility and safety constraints. To address these competing requirements, we present Feasible Action for Optimal Control (FAOC), a novel control framework integrating Reinforcement Learning (RL) and Optimal Control (OC). The key contribution is a computationally efficient, optimization-based mapping algorithm that transforms the RL agent's action from a static abstract set into a state-dependent feasible parameter set of the Optimal Control Problem (OCP), guaranteeing strict satisfaction of the dynamical system's constraints. Thus, FAOC effectively combines the predictable safety of OC with the flexibility of RL. In contrast to prior work, the abstract action space of the RL agent does not require expert or heuristic design, and the OCP formulation is not compromised by the inability of RL to guarantee feasibility. We apply our approach to real-time motion planning for robot table tennis, which encapsulates these challenges. Via simulated experiments, we show that FAOC outperforms state-of-the-art baselines in both sample efficiency and closed-loop performance.

Stefan Richter, Alberto Giammarino, Guillem Torrente· 2026-07-27T02:01:14

PaperSimulation仿真

Self-Supervised Consistency Enhanced Disentangled Learning for Neural Decoding Generalization in Brain-Machine Interface

Brain-Machine Interfaces (BMIs) provide a direct communication pathway between the brain and external devices, enabling humans to control assistive and robotic technologies, with potential applications in rehabilitation, human motor augmentation, and human-centered robotics. However, due to neural drift, the performance of BMIs decreases over time, posing challenges for long-term viability, particularly for invasive BMIs (iBMIs). Existing solutions suffer from two main drawbacks: (i) difficulty in learning robust neural representations, and (ii) neglecting that neural drift varies across motor parameters (e.g., velocity, direction, and speed). To overcome these limitations, we propose Self-Supervised Consistency enhanced Disentangled Learning (SSCDL), a neural decoding generalization framework built on two key innovations. We first design a backbone model named Consistency enhanced Neural Decoder (CND), using a novel teacher-student consistency constraint with simulated neural signal perturbations to learn robust representations invariant to neural drift. Then, we employ three dedicated CNDs under the Complementary-Disentangled Generalization (CDG) mechanism, which disentangles motor signals into velocity, direction, and speed with inspiration from neural preference theory. This disentangled learning enables SSCDL to capture invariant neural representations from diverse neural preference perspectives, significantly enhancing cross-day generalization. Extensive experimental results show that SSCDL delivers state-of-the-art decoding performance, exhibiting high robustness and cross-day stability. These capabilities underscore its strong potential for long-term interaction in human-centric robotic and fine-grained assistive applications.

Jiyu Wei, Di Hong, Zhanjie Zhang· 2026-07-27T05:40:01

PaperSimulation仿真

Moving-Horizon Estimation and Nonlinear Model Predictive Control of Cable-Driven Soft Manipulators

Precise control of soft manipulators remains challenging due to the difficulty of developing accurate yet computationally tractable models for model-based estimation and control. Reduced Cosserat-rod models provide a physics-based and control-oriented description of soft-robot dynamics, offering an explicit alternative to purely data-driven input-output representations. In this paper, we propose a moving-horizon estimation (MHE) and nonlinear model predictive control (NMPC) framework for cable-driven soft manipulators based on reduced Cosserat dynamics. A smooth cable-length-driven modeling formulation is developed by approximating the complementarity relationship between cable tension and cable slackness, enabling cable-length control without direct tension sensing. Based on this formulation, an MHE method is introduced to estimate the reduced state and reconstruct the manipulator configuration from end-effector pose measurements and cable-length information. An NMPC controller is then formulated to achieve task-space control under cable-length and cable-rate constraints. The proposed framework is validated through numerical simulations and experiments. Simulation results demonstrate the effectiveness of the estimator and controller for pose and strain-related regulation on a multi-cable soft manipulator. Experimental results on a four-cable prototype further show that the proposed MHE-NMPC scheme can be implemented in real time and enables accurate end-effector position tracking through cable-length control.

Lingxiao Xun, Haihong Li, Gang Zheng· 2026-07-27T05:54:39

PaperSimulation仿真

Effective Parameters, Real Behavior: Renormalization for Robotics -- From Infinite Electron Mass to Sim-to-Real Gap

Bridging the sim-to-real gap is a central problem in robotics, and the prevailing approach is to build increasingly accurate simulators. Here, we propose another approach based on renormalization: using effective, resolution-dependent parameters to absorb details omitted by the simulator and reproduce real behavior. These parameters may differ from measured physical values because they compensate for what the simulator leaves out. We demonstrate this mechanism analytically for proportional--derivative (PD) control at finite simulation frequency, where proportional feedback changes the effective derivative gain and derivative feedback changes the effective inertia. We then interpret dynamic rope manipulation and underwater swimming through the same perspective. Finally, we present a practical procedure for choosing observables, identifying omitted physics, and determining effective parameters. Renormalization offers robotics a complementary path across the sim-to-real gap: effective parameters, real behavior.

Youran Sun, Jiaxuan Guo, Xingyu Ren· 2026-07-27T07:20:30

PaperSimulation仿真

Learning Adaptive Multi-Task Guidance, Navigation, and Control via Hypernetworks

Autonomous free-flying robots in orbital environments require controllers that are both versatile and resource-efficient, yet maintaining a separate, task-specific policy for each mission profile is architecturally brittle and limits operational flexibility as requirements evolve. We introduce HYPER-GNC, a multi-task reinforcement learning framework in which a hypernetwork maps physics-informed task embeddings to the weights of a shared actor-critic policy, enabling a single compact controller to master four distinct GNC tasks: velocity tracking, docking, inspection, and navigation with obstacle avoidance. The continuous embedding space allows the controller to generalize to novel mission configurations at deployment time without any retraining. Extensive experiments demonstrate that HYPER-GNC achieves sample efficiency comparable to single-task specialists while maintaining stability under significant inertial perturbations and external body wrenches. We further validate the framework on a physical satellite emulator, successfully bridging the simulation-to-reality gap across all mission profiles. Code, trained models, and deployment scripts are made publicly available to support reproducibility.

Ricard Marsal I Castan, Aman Arora, Antoine Richard· 2026-07-27T11:39:22

PaperSimulation仿真

KAI: A Kinematic-Aware Interface for Data-Efficient Articulated Object Manipulation

Articulated object manipulation requires an understanding of kinematic structure that is difficult and costly to learn from robot demonstrations alone. We introduce the Kinematic-Aware Articulation Interface (KAI), a structured intermediate representation that captures the kinematic structure of articulated objects. By embedding interpretable geometric and kinematic priors into policy learning, KAI provides a strong inductive bias aligned with the underlying structure of articulated motion. This design effectively improves sample efficiency, with gains particularly pronounced in low-data regimes: across six simulation tasks, our method achieves an average success rate of 82.9%, matching or surpassing baseline performance while using only half the demonstration data. Our method also exhibits robust generalization to unseen backgrounds and visual distractors, transferring from a single clean training environment to cluttered real-world scenes. KAI's action-agnostic design further enables co-training with human interaction videos to enhance real-world robustness: under diverse visual distractions, our method with video co-training achieves over 70% average success rate.

Yaping Li, Zhaxizhuoma, Qiaojun Yu· 2026-07-27T14:29:47

PaperSimulation仿真

Explainable Reinforcement Learning via Physics-Aware Policy Distillation

In safety-critical sectors such as robotics and automotive engineering, the deployment of Deep Reinforcement Learning (DRL) is often hindered by the black-box nature of deep neural networks. This lack of transparency poses significant challenges for regulatory compliance and human-agent trust. This paper presents an experimental study aimed at making high-performance continuous control DRL systems interpretable. A policy distillation framework is implemented using the classic Inverted Pendulum benchmark. A high-performance Twin Delayed DDPG (TD3) agent serves as an opaque, continuous teacher model, whose policy is distilled into an interpretable student surrogate based on a shallow Decision Tree. By leveraging a custom physics-aware feature and "Noisy Oracle Rollouts" for dataset generation, the distillation process achieves performance equivalent to the expert teacher. Furthermore, comparative control theory analysis reveals a fundamental trade-off: transitioning from continuous to discrete rule-based control induces high-frequency Bang-Bang actuation and a stable bimodal limit cycle. Simulation results indicate that Bounded-Input Bounded-Output (BIBO) stability is maintained while providing both global and local interpretability for safe autonomous systems.

Shaker Al-Tamari, Waled Kadour· 2026-07-27T17:14:42

PaperSimulation仿真

Exploration Matters for Escaping the Blur Trap in 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) employs Gaussian primitives for explicit scene representation, facilitating real-time, high-fidelity reconstruction and novel view synthesis of complex scenes. However, the explicit modeling inherent in 3DGS introduces a gradient bias during optimization, rendering its non-convex optimization process highly susceptible to convergence toward local suboptimal solutions. This constitutes a fundamental limitation in 3DGS optimization, which we term the Blur Trap. To address this limitation, we integrate simple explicit exploration into the 3DGS optimization framework. First, through rigorous mathematical analysis of the 3DGS optimization formulation, we identify the underlying optimization bias responsible for the Blur Trap and categorize it into two distinct subtypes: the Far-Side Blur Trap and the Near-Side Blur Trap. Subsequently, we propose two highly straightforward exploration strategies (Random Seeding and Random Splitting) to mitigate the far-side and near-side blur traps, respectively. Experimental validation demonstrates that the incorporation of these exploration operators effectively and complementarily overcome the Blur Trap, achieving high-quality rendering performance across multiple datasets. Project page: https://chengbo-wang.github.io/ExploreGS/

Chengbo Wang, Guozheng Ma, Jinhong Wu· 2026-07-20T14:05:54

Paper3D Gaussian Splatting3DGS

QIRF Quantum-Inspired Non-Orthogonal Function-Space Compression for 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) achieves high-quality real-time rendering by representing a scene with a large collection of anisotropic Gaussian primitives. However, complex scenes often require millions of Gaussians, resulting in substantial storage and rendering costs. Existing compression methods mainly reduce redundancy through primitive-wise pruning, attribute quantization, clustering, or neural coding, while redundancy caused by strongly overlapping and non-orthogonal Gaussian basis functions remains largely unexplored. We present QIRF, a quantum-inspired non-orthogonal function-space compression method for 3D Gaussian Splatting. QIRF models neighboring Gaussian primitives as a local non-orthogonal basis and formulates primitive reduction as a subspace-aware selection problem. Specifically, an analytic Gaussian overlap matrix and a radiance-response density matrix are constructed to characterize functional redundancy and rendering relevance. Generalized eigendecomposition is then used to identify the dominant local subspace and select representative Gaussian primitives. An RRDM-based response model and detail-aware safeguarding further preserve visually important high-frequency structures under aggressive pruning. Experiments on 13 scenes from Mip-NeRF 360, Tanks and Temples, and Deep Blending show that QIRF reduces the Gaussian count and raw PLY storage by 71.7 percent on average, corresponding to approximately 3.54 times compression, while maintaining reconstruction quality comparable to 3DGS and achieving a marginal average PSNR improvement of 0.10 dB. QIRF also improves the average rendering speed over 3DGS by 34.3 percent. These results suggest that non-orthogonal function-space redundancy is an important yet underexplored source of representational redundancy in explicit Gaussian radiance fields.

Shizeng Jiang, Hao Zhang, Xuerui Ma· 2026-07-20T15:34:45

Paper3D Gaussian Splatting3DGS

AniGS: Bridging Rendering and Diffusion Prior for 3D Scene Animation

Novel view rendering of large and complex reconstructed scenes is becoming increasingly photorealistic. However, most reconstructions remain static and lack the ambient motion that makes environments immersive. We present AniGS, a method for scene-level animation of 3D Gaussian Splatting (3DGS) reconstructions that adds subtle, distributed dynamics, e.g., vegetation motion, while preserving rigid structures. Unlike existing 3D animation techniques which are limited to object-centric subjects or small regions, AniGS is designed for large, cluttered, navigable scenes. AniGS represents the scene with a canonical 3DGS and models motion using a time-conditioned deformation field. To animate the entire scene, we leverage a pretrained video diffusion model and introduce an iterative dataset--model update strategy that progressively expands viewpoint coverage and repeatedly updates camera-fixed training videos using a render-and-refine scheme. To prevent artifacts from unintended motion in static areas, we further introduce a composed video-to-video refinement scheme that restricts motion to desired regions. Experiments on five real-world, large-scale outdoor scenes demonstrate that AniGS produces natural ambient dynamics and high-quality novel view videos, enabling more immersive viewing experiences of reconstructed environments.

Yen-Chi Cheng, Chen Gao, Chuhan Chen· 2026-07-20T22:14:03

Paper3D Gaussian Splatting3DGS

ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting

Recent advancements in 3D Gaussian Splatting (3DGS) have enabled language-guided scene understanding. However, existing Referring 3D Gaussian Splatting (R3DGS) methods are fundamentally restricted to single-target queries. To reflect the ambiguity of real-world instructions, we introduce the Generalized Referring 3D Gaussian Splatting Segmentation (GR3DGS) task, which requires dynamically segmenting an arbitrary number of targets (0, 1, or $N$). To facilitate comprehensive evaluation of this new task, we construct two new benchmarks: GR-LERF and GR-ScanNet. Crucially, existing R3DGS paradigms exhibit fundamental technical bottlenecks that severely limit their performance on the GR3DGS task: they lack intrinsic 3D point-level understanding by operating merely on 2D rendered pixels, and they incur prohibitive computational overhead by requiring per-scene optimization to embed heavy semantic features. To dismantle these bottlenecks, we propose ZeroSplat, a novel training-free and zero-feature framework. ZeroSplat lifts 2D Vision-Language Model (VLM) priors into 3D space through robust multi-view geometric constraints. This strategy enables intrinsic point-level understanding without incurring any additional feature storage. Extensive experiments demonstrate that ZeroSplat significantly outperforms state-of-the-art methods across generalized and single-target scenarios while maintaining exceptional efficiency. Project Page: https://inkmind-ai.github.io/ZeroSplat

Jiayu Ding, Meilu Song, Xiaoyi Zhang· 2026-07-21T07:27:10

Paper3D Gaussian Splatting3DGS

TOM-GS: Editable Video Representation via Temporal Opacity Modulation of Static 3D Gaussians

While Implicit Neural Representations (INRs) and dynamic 3D Gaussian Splatting (3DGS) achieve impressive results in video processing, they often fall short of producing representations that are easily editable. Recent methods address this by introducing complex spatial deformations or folded distributions, which constrain optimization and reduce flexibility for downstream editing. In this paper, we introduce TOM-GS, an editable video representation that forgoes complex deformations in favor of regular 3D Gaussians equipped with a continuous temporal opacity formulation. By assigning a learnable temporal mean and scale to the opacity of each Gaussian, our model enables static 3D spatial components to fade smoothly in and out of the scene. Grounded by robust, off-the-shelf pose estimation, our approach maintains a static spatial geometry that naturally supports a wide range of manual and physics-based edits. TOM-GS outperforms prior editable video representations in visual fidelity, while its reliance on standard 3D Gaussians ensures seamless compatibility with established 3D editing tools.

Marek Lisowski, Łukasz Smoliński, Kornel Howil· 2026-07-21T16:27:32

Paper3D Gaussian Splatting3DGS

RealVDeblur: One-Step Diffusion for Generalizable Real-World Video Deblurring

Real-world video deblurring remains challenging due to diverse motion patterns, complex degradations, and the scarcity of realistic training data, yet robust restoration is critical for downstream pipelines such as mobile imaging and 3D reconstruction. This work presents \textbf{RealVDeblur}, an efficient generative framework designed to improve in-the-wild robustness under diverse real capture conditions. First, a large-scale, physically grounded blur synthesis pipeline is constructed from scene-level 3D Gaussian Splatting (3DGS) assets and high-frame-rate videos, providing realistic training data covering both camera-induced and object-motion blur. Second, a video diffusion prior is leveraged for restoration; to better accommodate frame-dependent blur variations, temporal compression in the VAE is disabled and a frame-wise encoding scheme is adopted. For practical deployment on long videos, multi-step diffusion sampling is distilled into an efficient one-step generator, and a training-free Temporal Window Mask stabilizes inference beyond the training horizon with constant memory usage. Extensive experiments on diverse real-world benchmarks demonstrate strong perceptual quality, semantic fidelity, and temporal consistency on unseen videos, as well as improved robustness in downstream 3D reconstruction under severe motion blur. Project page: https://rbjin.github.io/RealVDeblur

Renbiao Jin, Mingxin Yang, Yutian Chen· 2026-07-22T18:01:03

Paper3D Gaussian Splatting3DGS

Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis

Visual localization becomes extremely challenging in planetary-like terrains characterized by low texture, perceptual aliasing, harsh illumination, and sparse, weakly overlapping viewpoints induced by forward rover motion and unconstrained driving directions. Under these conditions, state-of-the-art image-to-image and image-to-map matching pipelines suffer significant performance degradation. In this work, we propose a visual relocalization method that departs from classical correspondence-based pipelines by directly estimating camera poses against a differentiable map representation built with 3D Gaussian Splatting (3DGS). Our key contribution is a geometry-aware training strategy that combines photometric and geometric losses, where the geometric supervision is provided for the first time by combining multi-view stereo (MVS) and LiDAR depths. We show that this joint optimization produces a 3DGS model that better fits the underlying scene geometry, leading to improved photometric and geometric consistency and more robust, accurate single-image 6-DoF pose estimation. Extensive experiments on data acquired in planetary-analog environments validate the effectiveness of our approach, showing substantial gains in relocalization accuracy under challenging conditions. Code is available at https://github.com/DLR-RM/multimodal-gsplat-relocalization.

Maria Peribañez, Javier Civera, Rudolph Triebel· 2026-07-24T09:46:07

Paper3D Gaussian Splatting3DGS

Meshless Domain Randomization via Explicit Parameter Perturbation of 3D Gaussian Splatting

Domain Randomization (DR) is a standard technique for closing the Sim-to-Real gap, yet traditional DR pipelines rely on classical computer graphics rendering driven by polygon meshes. For complex organic subjects, such as insect specimens, extracting and rendering textured meshes is challenging. To address this issue, we propose a meshless DR framework that operates on the parameter space of 3D Gaussian Splatting (3DGS). Our method employs two independent perturbation pipelines to synthesize randomized training datasets. First, a Photometric DR pipeline alters the baked illumination and color balance by modulating the Spherical Harmonics (SH) coefficients. Second, a Procedural DR pipeline isolates the subject's geometric shape by replacing its original textures with 3D spatial noise. Finally, these perturbed radiance fields are composited over stochastically varied backgrounds using a rasterization engine. Our parameter manipulation provides a meshless alternative for generating robust datasets for complex geometries.

Felipe Nunes Carbone de Carvalho, Joyce de Morais Souza, Alan de Aguiar· 2026-07-24T20:01:33

Paper3D Gaussian Splatting3DGS

Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design

AI-generated content (AIGC) has made significant progress, with 2D generative models becoming ready-to-use tools for the digital fashion industry. However, 3D garment generation remains in its nascent stage, where in the realm of fashion, the semantic information of diverse design elements exhibits intricate coupling relationships in 3D representations, posing substantial challenges for generating diverse 3D garments. In this work, to handle the above problem, We introduce Fashion-3DLR, a novel 3D garment generation framework that utilizes diverse design elements to create high-quality, versatile 3D garment assets. Specifically, to bridge the semantic gaps between different fashion elements, we propose a Garment Feature Fusion Diffusion Transformer (GFF-DiT) module to integrate 2D fashion design elements, e.g., sketch and texture, into latent space. Within the latent space, we then employ a rectified flow transformer to generate geometry latents, which can be decoded into various 3D garment representations, including 3D Gaussians and meshes. Furthermore, we integrate Fashion-3DLR into downstream tasks, achieving the 3D Gaussian Splatting (3DGS)-driven cloth physical simulation and mesh-based virtual try-on. Experimental results indicate that Fashion-3DLR surpass the previous state-of-the-art methods, which verify that the proposed work can generate well-structured, non-watertight garments capable of physical simulation and virtual try-on, underscoring its potential as a versatile 3D garment design tool.

Shenghao Yang, Hongtao Zhang, Yuhan Yi· 2026-07-25T13:10:38

Paper3D Gaussian Splatting3DGS

Head Avatars with Dynamic Explicit Hair

We present DynHair, a novel method for tracking and modeling dynamic hair for human head avatars. From video input, we reconstruct a dynamic head avatar with an explicit strand-based hair representation using structured 3D Gaussian Splatting. In contrast to the face region of human head avatars, which can be modeled with 3D Gaussians that are attached or generated with respect to some expressive 3D head model, hair is particularly challenging as it exhibits dynamic motion effects. Therefore, we present a novel method that models the dynamic deformations of the hair strands using a temporal network that is conditioned on angular velocity and acceleration of the head, as well as relative gravity. Specifically, an LSTM encodes the motion history and modulates per-point strand features via FiLM conditioning which further used by MLP to produce physically plausible displacements to canonical hairstyle. We jointly optimize this motion and appearance representation of the hair, with a 3DGS-based representation of the face-region, via differentiable Gaussian splatting with photometric, geometric, and physics-based supervision. As a result of our method, we retrieve hair tracking of the training video data and an animatable head avatar with controllable hair dynamics. In our experiments, we demonstrate state-of-the-art performance in terms of hair dynamics, temporal consistency, and generalization across subjects.

Vanessa Sklyarova, Haonan Chen, Berna Kabadayi· 2026-07-26T22:08:13

Paper3D Gaussian Splatting3DGS

GenSplatCodec: Feed-Forward Gaussian Splatting Compression via One-Step Diffusion

Feed-forward 3D Gaussian Splatting (3DGS) enables scalable scene reconstruction without per-scene optimization, yet produces dense Gaussians that are costly to store and transmit. Existing feed-forward Gaussian compression methods formulate decoding as deterministic representation recovery, which becomes inadequate at low bitrates when high-frequency textures and view-dependent appearance are discarded. Although generative models offer a promising alternative, using them as standalone post-processing decouples generation from the transmitted scene structure, thereby compromising cross-view consistency. To address these limitations, we propose GenSplatCodec, a unified feed-forward Gaussian codec that reformulates low-bitrate Gaussian compression as geometry-guided generative decoding. We present a detail-aware feed-forward Gaussian coding scheme within a dual-stream formulation, where the resulting compact Gaussian structural stream is complemented by a lightweight reference appearance stream. We further introduce a geometry-guided one-step generative decoding approach that jointly exploits decoded structural and appearance cues through hierarchical geometry control to reconstruct high-fidelity and view-consistent novel views. Finally, we develop a three-stage optimization strategy that stabilizes the learning of the unified codec and adapts the generative decoder to codec-derived structural and appearance cues. Extensive experiments across multiple datasets demonstrate that GenSplatCodec consistently achieves superior rate-distortion (RD) performance over existing methods.

Qiang Hu, Zhenlong Wu, Lei Huang· 2026-07-27T13:20:26

Paper3D Gaussian Splatting3DGS

FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation

The sense of touch is central to manipulation, especially when vision is occluded or ambiguous. Although combining vision and touch improves manipulation, learning robust visuo-tactile policies requires substantial tactile data. Such data remains scarcer than visual data, because tactile sensors are fragile, specialized, and hard to standardize. To address this, we present Feature-Extracted Latent Tactile (FELT), a learning-based framework that synthesizes per-finger pressure tactile images from RGB observations, reducing the need for tactile-equipped data collection. FELT uses a large frozen visual encoder and a lightweight query decoder to predict tactile signals in a single feed-forward pass. To respect the physical topology of dual-finger tactile sensors, FELT decodes the left and right tactile sensor panels through separate branches, capturing the asymmetric contact patterns during interactions such as wiping, insertion, and in-hand rotation. At inference time, FELT only requires RGB data, allowing us to augment existing vision-only data with tactile observations, either as generated tactile images or as latent tactile features. Experiments on four contact-rich manipulation tasks demonstrate that both generated tactile images and latent tactile features improve policy success over vision-only baselines, with latent feature requiring no real tactile sensor during policy training or deployment. Supplementary material is available on our anonymous website: https://felt-tactile.github.io/.

Zinan Li, Yiyang Ling, Yuming Gu· 2026-07-22T19:32:42

触觉生成视触觉跨模态操作Paper

Dynamics-Aware Meta-Imitation for Generalization to Unseen Robotic Manipulation

Imitation Learning aims to learn skills from extensive observations and demonstrations for robots, so it suffers from data scarcity and environment generalization. The existing methods predominantly focus on imitation from in-domain tasks and consequently struggle with generalization to unseen tasks. To bridge this generalization gap, we propose the \textbf{D}ynamics-\textbf{A}ware \textbf{M}eta-\textbf{I}mitation (DAMI) framework. By integrating meta-learning to construct a shared skill space, DAMI equips agents for rapid adaptation to novel tasks. We introduce the Visual-Motor Trajectory (VMT) module to capture complex spatio-temporal dynamics within the task latent space. Furthermore, we propose the Unpaired Unified Task (U2T) block to fuse unstructured multimodal observations. To coordinate these representations, we integrate a Task-Conditioned Feature Modulation (TCFM) mechanism customized for modulating low-level 3D features. By capturing intrinsic dynamics from a random complete reference demonstration, our framework learns the underlying task logic rather than memorizing static cues, ensuring effective generalization. Extensive experiments in both simulation and real-world settings demonstrate that our approach outperforms state-of-the-art baselines regarding direct inference on seen tasks and adaptation to unseen tasks via few-shot fine-tuning.

Zhenduo Shang, Xiyao Liu, Bohan Li· 2026-07-17T11:50:03

模仿学习元学习泛化动力学操作

GuidedAttention: Interpretable and Correctable Visual Attention for OOD-Robust Robot Manipulation via Imitation Learning

End-to-end visuomotor policies provide little opportunity for humans to understand or correct the policy's visual attention. We propose GuidedAttention, a visuomotor imitation learning framework that introduces interpretable and correctable visual attention as an explicit intermediate representation. Task-relevant attention keypoints are predicted from camera images and condition a diffusion-based action policy. Users can inspect and optionally correct selected keypoints once at rollout initialization, after which the corrected attention is automatically propagated throughout execution by a tracking module. Experiments in simulation and the real world demonstrate that GuidedAttention consistently improves robot manipulation performance, particularly under positional and appearance out-of-distribution (OOD) conditions.

Masaki Murooka, Ryoichi Nakajo, Keisuke Shirai· 2026-07-23T08:33:40

模仿学习视觉注意力OOD可解释操作

Look Before You Edit: Attention-Guided Camera Placement and Multi-View Alignment for 3D Gaussian Splatting Editing

Text-driven 3D scene editing with 3D Gaussian Splatting (3DGS) typically applies a 2D diffusion editor to views rendered from fixed training cameras, limiting both the spatial coverage of edits and the user's freedom to target specific objects in complex scenes. We present LB-Edit, a framework that addresses two coupled problems: where to place editing cameras for localized edits, and how to make per-view edits agree with one another so that the 3D scene remains consistent after fine-tuning. First, Attention-Guided Editing Camera Placement (ACP) probes the diffusion model's self- and cross-attention at multiple candidate camera distances to find where attention is well-contained in the region of interest, then places a compact, geometrically diverse editing camera set at that attention-optimal distance. Second, Multi-view Attention Alignment (MAA) steers the editor toward the same edit across views along two axes: it aligns appearance by sharing self-attention features via token-level correspondence, and aligns spatial location by lifting cross-attention maps onto the 3D Gaussians as a shared 3D attention field, suppressing both appearance and spatial drift. Experiments on multi-object and single-object scenes show that our method achieves the highest user preference in instruction fidelity, multi-view consistency, and editing locality, using as few as 5 editing views and reducing latency by up to 7x over existing methods.

Jaeyeon Park, Taeho Kang, Youngki Lee· 2026-07-22T05:51:10

3DGS场景编辑相机放置多视角Paper

ATSplat: Compact Feed-forward 3D Gaussian Splatting with Adaptive Token Expansion

3D Gaussian Splatting (3DGS) achieves high-quality novel-view synthesis by optimizing freely placed primitives in 3D and adaptively densifying them in under-reconstructed regions. However, this scene-adaptive capacity allocation is largely lost in existing feed-forward 3DGS methods, which commonly regress Gaussians at input pixels and lift them along camera rays. Such pixel-aligned formulations make the number and placement of primitives depend on image resolution and input viewpoints rather than scene complexity, resulting in dense and often redundant Gaussian sets. We present ATSplat, a feed-forward 3DGS framework that restores the adaptive allocation capability of 3DGS optimization through Adaptive 3D Tokens. ATSplat first lifts coarse patch-level depth and camera cues into sparse 3D anchor tokens, forming a compact scaffold of the scene. Each token is then regressed into local Gaussians with learnable 3D offsets, decoupling primitive placement from input image grids. An Adaptive Token Expansion module predicts a token-level uncertainty score, supervised by rendering error maps, and selectively expands high-uncertainty tokens through learnable expansion layers. This sparse-to-adaptive formulation enables ATSplat to concentrate primitives in challenging regions while maintaining a compact representation. Experiments on two representative datasets, RealEstate10K and DL3DV, show that ATSplat achieves state-of-the-art rendering quality while reducing the number of Gaussians by more than $5.7\times$ compared with dense feed-forward 3DGS methods. From 12 input images at $512 \times 960$ resolution, ATSplat completes reconstruction in less than a second using a single commercial GPU, and renders high-quality novel views at 1136 FPS ($512 \times 960$) with only 311K Gaussians.

Cho In, Jeonghwan Cho, Mijin Yoo· 2026-07-22T17:54:44

3DGS前馈token自适应紧凑

SubSplat: High-Resolution Pixel-aligned 3DGS via Sub-pixel Gaussian Reparameterization

Pixel-aligned Gaussian splatting enables efficient and generalizable novel-view synthesis. However, high-resolution rendering faces a critical trade-off where increasing input resolution improves detail at the expense of quadratically rising network computational cost. Conversely, maintaining low-resolution inputs stabilizes this cost but results in insufficient Gaussian density and artifacts. To address this, we propose SubSplat, which introduces Sub-pixel Gaussian Reparameterizer(SPGR) to subdivide primary Gaussians into fine-grained primitives, restoring structural density directly from low-resolution features. We further enhance the reparameterization quality through feature aggregation, which effectively captures high-frequency details across multiple views. Experiments on RealEstate10K and ACID demonstrate that SubSplat achieves high-fidelity rendering with superior efficiency. Our results validate that the proposed framework successfully resolves the trade-off between reparameterization fidelity and network computational cost inherent in pixel-aligned Gaussian Splatting.

Jiun Lee, Jaekwang Kim, Sangmin Lee· 2026-07-23T00:56:18

3DGS亚像素高分辨率重参数化Paper

GLAM-SLAM: Real-time Gaussian Large-scale Mapping via Flow Densification and Spatial Decomposition

Existing Gaussian-splatting-based monocular Simultaneous Localization and Mapping (SLAM) systems are either tailored to short sequences, are not real-time, or suffer from prohibitive GPU memory requirements, limiting their applicability in realistic, long-horizon scenarios. To address this, we present GLAM-SLAM, a real-time, decoupled Gaussian-splatting SLAM system designed for large-scale outdoor scenes. We ensure lightweight tracking using a robust, feature-based SLAM frontend, while for mapping, we adopt a structured, sparse anchor grid representation that ensures scalable operation and maintains scene coherence across long-term sequences. To satisfy the dense initialization requirements of 3D Gaussian Splatting (3DGS), we introduce a geometry-based flow-densification anchoring strategy using epipolar constraints. Furthermore, by treating mapping as a multi-scene problem, we propose a scene-partitioning strategy that introduces a strong spatial inductive bias via MLP initializations to generate localized Gaussians. We evaluate our system on the challenging, long-sequence KITTI Odometry, Oxford RobotCar, and M'alaga datasets. Extensive ablations and comparisons demonstrate a 15% improvement in reconstruction quality over the second-best performer, while maintaining real-time performance and the ability to scale to longer sequences. Code is publicly available for the benefit of the community.

Panagiotis Mermigkas, Argyris Manetas, Petros Maragos· 2026-07-23T15:22:42

3DGSSLAM实时大规模建图

GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis

Dynamic scene reconstruction with 3D Gaussian Splatting requires a balance between fine-grained motion modeling, structural stability, and compact representation. Existing per-primitive methods provide flexible local deformation but often suffer from redundant primitive growth, while anchor-based methods improve spatial regularity at the cost of suppressing locally varying motion. To address these issues, we present GrainGS, a dynamic Gaussian framework that combines a hierarchical anchor scaffold with per-Gaussian deformation. A static warm-up stage first establishes a time-invariant canonical representation from observations across all timestamps. During joint training, a stop-gradient operation blocks the deformation-mediated gradient pathway to the canonical positions while preserving their direct refinement through the reconstruction objective. Each Gaussian then predicts independent temporal offsets for position, rotation, and scale, enabling detailed local motion within a structurally constrained scaffold. A canonical-residual appearance decomposition further models frame-dependent photometric changes without forcing them into geometric deformation. Experiments on synthetic monocular and real-world multiview benchmarks show that GrainGS achieves high reconstruction quality, real-time novel view synthesis, and compact storage. Under the synthetic benchmark setting, it reaches an average peak signal-to-noise ratio of 36.98 decibels, renders at 435.6 frames per second, and requires 4.67 megabytes of storage.

Jiahao He, Yihua Shao, Zhengkai Zhao· 2026-07-23T15:52:35

3DGS动态场景新视角合成梯度解耦Paper

GS-Agent: Creating 4D Physical Worlds With Generative Simulation

Creating dynamic and physically realistic 4D worlds from natural language descriptions is both fascinating and challenging. Traditional computer graphics methods rely on manual creation, requiring extensive human effort to fine-tune materials, motions, and visual fidelity. Recent advances in generative foundation models have sparked interest in learning to generate such 4D worlds from large-scale data; however, existing methods still struggle to ensure physical plausibility and controllability. In this work, we take a different path by leveraging foundation models to construct an agentic system that emulates how humans traditionally create 4D worlds, yet automates the entire process. We present GS-Agent, an end-to-end multi-agent framework that integrates physics engines in the loop to generate realistic, dynamic, and controllable 4D physical worlds from natural language. Inspired by how humans build 4D worlds, GS-Agent decomposes the task into entity management, covering 3D asset curation, material tuning, placement, and motion control, and rendering configuration, including camera and lighting manipulation. Multiple agents with distinct expertise interact with the physics engine via code, seek multimodal feedback, and collaborate to iteratively construct 4D worlds that align with the given descriptions. Experimental results show that GS-Agent effectively converts natural language into diverse and physically plausible 4D worlds exhibiting rich interactions among liquids, deformable objects, and rigid bodies, while achieving cinematic camera and lighting control. We envision GS-Agent as a foundation for a new paradigm in 4D world generation, empowering creative content creation and physical AI. Project page at https://umass-embodied-agi.github.io/gs-agent/

Hongxin Zhang, Chunru Lin, Junyan Li· 2026-07-23T17:04:36

4D生成世界模型3DGS物理仿真生成式

World Translation: Minimizing Sim-to-Real Gap with Backward Dynamics Extraction and Unpaired Domain Translation

The gap between simulation and reality remains a fundamental challenge in deploying simulation-trained robotic policies in the real world. Real-to-sim methods narrow this gap from the real side, learning transition dynamics from real data to build a more realistic digital world. Learned dynamics models are their dominant instance. Such methods, however, face a partial observability problem: the same observation may branch to different transitions due to unobservable factors. Existing methods assume these factors can be recovered from observation history. However, this may fail whenever observation history is uninformative, such as a sudden contact event with no prior warning. To address this limitation, we propose \textit{World Translation}, which exploits a complementary strength of simulators and learned dynamics. Simulators are deterministic but physically imperfect, while learned models are accurate but underdetermined under partial observability. Rather than predicting transitions forward from history, we extract the unobservable dynamics information backward from an observed transition, then translate this feature across simulation and reality as an unpaired domain-translation problem that preserves dynamics content while transferring domain style. Experiments across humanoid, quadruped, and manipulator platforms show that our method achieves more accurate dynamics modeling than baselines, with the largest gains when unobservable factors cannot be recovered from observation history. Real-robot deployment on Go2 quadruped confirms improved policy transfer.

Xinchen Yao, Leixin Chang, Hua Chen· 2026-07-20T16:52:58

sim-to-real域适应世界模型反向动力学Paper

Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents

Real-to-sim conversion for robotic interaction with objects remains labor-intensive because it requires more than visual reconstruction: a streamlined real2sim process must recover scene geometries and object states, infer physical parameters, and assemble actors, objects, cameras, poses, and trajectories into a runnable physical simulation. Today this process still depends on manual tuning of visual foundation models, mesh cleanup, coordinate-frame alignment, and brittle workflow glue across visual perception tools and simulators. We introduce \textit{Agentic Real2Sim}, a framework for generalized physical world modeling with vision-language agents, converting a real-world recording of object-robot interaction into a simulatable episodic twin which preserves observations, geometries, robot interactions, and object states. We evaluate Agentic Real2Sim on rigid-object manipulation, deformable-object interaction, and humanoid motion scenes, spanning domains that are usually handled by separate Real2Sim pipelines, marking a first step toward scalable conversion. The framework's agentic decisions can be driven by an open-weight VLM backend at a small fraction of the cost of frontier models, while attaining comparable conversion success rate. We aim to use the resulting real-world-aligned twins for downstream robotics tasks, specifically policy learning and evaluation. The project site is available at https://agentic-real2sim.github.io/.

Guanxiong Chen, Qianjun Xia, Jiawei Peng· 2026-07-21T15:23:38

世界模型视觉语言模型sim-to-real物理仿真Paper

Odin: Primitive-Level Synchronization for Distributed Point-Based Neural Rendering

Point-based neural rendering (PBNR) represents 3D scenes as explicit, trainable primitives and underpins high-quality reconstruction and emerging embodied AI and world-model pipelines. Unlike layer-structured neural networks, PBNR has primitive-indexed dependencies: each view reads and updates only a sparse, view-dependent subset of mutable scene state. As large scenes require distributed training and optimized renderers reduce per-view computation, global task- or iteration-level barriers increasingly place synchronization, rather than rendering, on the critical path. We present Odin, a distributed PBNR training system that replaces global barriers with primitive-level synchronization. Its ahead-of-time scheduler uses stable locality and phase order to identify low-conflict overlap windows, while the runtime validates primitive publication before later work observes mutable state. Odin provides a quality-first path that preserves synchronized-training visibility and a throughput-first path that uses overlap and gradient evidence to admit only small, low-impact delayed reads; structural changes and high-impact cases remain synchronized. Across four existing PBNR pipelines and 13 non-city scenes on 8 GPUs, Odin improves throughput by 1.22 times on average and hides 82% of critical-path wait while preserving reconstruction quality. In a MatrixCity mixed-parallel case study scaling to 64 GPUs, Odin improves throughput over Grendel by up to 1.89 times without changing renderer kernels, optimizers, training budgets, or model capacity.

Zhenxiang Ma, Zeyu He, Yuanzhen Zhou· 2026-07-22T08:27:33

神经渲染3DGS点云分布式Paper

VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds

Vision and touch are complementary modalities essential for robotic perception and manipulation. While vision provides global object context, touch offers precise local information at contact points. Integrating these modalities for contact localization, i.e., predicting the location of touch on an object's surface, poses significant challenges due to the need for accurate spatial alignment between tactile data and visual geometry. To address this challenge, we propose VTLoc, a novel visual-tactile framework that localizes contact points from tactile readings using a 3D point cloud as visual input. VTLoc introduces two key components: a geometric multi-modal alignment module, which reconstructs a pseudo-point cloud from fused visual-tactile features and aligns it with the visual point cloud to enforce spatial consistencies across modalities; and an iterative localizing updater, which iteratively refines the predicted contact location using fused visual-tactile features. Evaluated on a new benchmark of 100 real-world objects, VTLoc improves single-touch contact localization by reducing local-to-global correspondence ambiguity.

Zhiyuan Wu, Zhuo Chen, Shan Luo· 2026-07-17T17:27:08

触觉感知tactile视觉点云point cloud

Think at 5 Hz, Act at 20 Hz: Asynchronous Fast-Slow Vision-Language-Action Inference for Closed-Loop Driving

Large language models bring instruction following and scene reasoning to end-to-end driving, but their inference latency collides with the control rate a vehicle requires. Existing closed-loop agents hide this gap by invoking the model on alternate simulation ticks and replaying the previous command in between, so half of all control outputs ignore the newest observations. We present a fast-slow architecture that removes this compromise. A frozen 7B vision-language backbone acts as the slow system, digesting navigation instructions and visual history at low frequency while exposing its per-layer key-value cache as a standing representation of the scene. A lightweight action expert acts as the fast system, attending to this cache and to the current camera frame at every simulation tick to regress waypoints in a single forward pass. Since the cache lags behind the world at deployment, we train the expert under randomized staleness, aligning training with asynchronous execution. On LangAuto-Short routes in CARLA, our system produces fresh control at every 50 ms simulation tick and lifts route completion from 37.0 to 94.0 over the frame-skipping baseline. A frame-skip ablation with the same expert separates the two factors at work: the expert raises the driving score on its own, while per-tick freshness raises completion from 82.1 to 94.0 and cuts red-light violations by a third. Trained on a single town, the expert transfers zero-shot to two unseen towns, holding 84-94% route completion where the baseline reaches 31-41%. It reduces open-loop waypoint error by nearly a factor of four compared to the backbone's own action head, at a per-tick model cost of 32 ms that is independent of history length on a single consumer GPU.

Yun Li, Jiachen Gong, Simon Thompson· 2026-07-17T04:40:02

自动驾驶autonomous driving仿真simulationtesting

AC-VLA: Robust Out-of-Distribution Action Execution via Compositional Learning

Vision-Language-Action (VLA) models excel at end-to-end robotic manipulation but struggle with out-of-distribution (OOD) generalization when familiar sub-tasks are recombined in unseen configurations. We identify two mutually reinforcing failure modes: \emph{trajectory overfitting}, where models overfit to holistic trajectory patterns rather than compositional sub-skill semantics; and \emph{perceptual shortcut}, where action tokens over-rely on wrist-view textures at the expense of global spatial grounding. To address both, we introduce \textbf{AC-VLA}, a plug-and-play Action Compositional learning framework comprising two architecture-agnostic components: \textbf{(i)} a compositional learning module that uses an LLM-driven instruction decomposer and a proprioceptive trajectory aligner to generate dense sub-task supervision, followed by mixed training on complete demonstrations and decomposed data to endow the model with compositional generalization; and \textbf{(ii)} a state-conditioned asymmetric masking strategy that suppresses wrist-view inputs during closed-gripper phases, enforcing global semantic grounding. All components are architectural modification-free and directly integrable into any VLA backbone. Instantiated on $π_{0.5}$ and evaluated on LIBERO and LIBERO-OOD benchmarks, AC-VLA achieves a ~28% absolute improvement on compositional OOD tasks while maintaining near-perfect in-distribution performance.

Xiaojiang Peng, Kai Peng, Jie Lu· 2026-07-17T07:51:03

触觉感知tactile仿真simulationmanipulation

DexPoint: Generalizable Point Cloud Policies for Dexterous Manipulation

DexPoint uses point cloud representations for generalizable dexterous manipulation without CAD models, showing strong sim-to-real transfer.

CMU, UC Berkeley

灵巧手点云操作sim-to-real开源

Blog (6)

Introducing S1: In-Context Learning for Robotics — One Video Demonstration Is All It Takes

2026-08-18T00:00:00

Skild AIS1上下文学习in-context learning机器人基础模型

Cross-Embodiment Transfer of a Frozen Humanoid Whole-Body Controller via Analytic Codec and LoRA Adapters

2026-08-26T00:00:00

cross-embodiment transferGEAR-SONICAgiBot X2Unitree G1LoRA

ADEPT: Pre-Train Dexterity Once, Post-Train Every Task

2026-08-19T00:00:00

灵巧操作强化学习sim-to-real预训练dexterous manipulation

Bringing 3DGS into NVIDIA Isaac Sim: A Photoreal Import Workflow and Four Industrial-Simulation Weaknesses

2026-07-26T00:00:00

3DGSIsaac Sim仿真环境USD语义分割

Setting Up a 3DGS Simulation Environment for Isaac Sim: Untouched Scan Data, Ground-Hole Patches, and a PhysX Vehicle in Three Steps

2026-08-16T00:00:00

3DGSIsaac Sim仿真环境LiDAR数字孪生

The Developer Inflection Point: Leju's Open-Source LeTools Post-Training Toolchain Compresses Embodied-AI Development from Weeks to Days

2026-08-15T00:00:00

具身智能LeToolssim-to-real行为树VLA

Physical AI

Videos (51)

CHC5 Open Camera: FPGA Vision Processing at 2,000 FPS

AGIBOT showroom and physical AI tour

Zetta ζ: Closed-Loop Recovery Harness on Frozen VLA Policies

Meta Reality Labs open-sources Project SuperDex, a contact-first dexterous manipulation simulator

Lightwheel Releases EgoSuite-Open100K: Open Egocentric Dataset for Physical AI

DexRobot open-sources SO101 Pick Cube SFT checkpoint and DM0.5 LoRA SFT workflow

SPD: VR Simulation Pre-training for Bimanual Dexterity

3D HAMSTER: 3D End-Effector Trajectory from RGB-D

Inside SuperMap: Persistent 3D Object Identities

SuperMap Turns SLAM into Persistent 4D Spatial Memory

Magenta centaur robot combines dual arms and a wheeled quadruped

NVIDIA SimFoundry turns real scenes into simulation worlds

DeWorldSG: Probabilistic 3D Gaussian Scene Graph Generation (ECCV 2026)

DJI and XGRIDS air-ground 3DGS map fusion

OpenAI sim-trained robot hand solves Rubiks cube

DJI M4E and PortalCam air-ground 3DGS map fusion

piR2: real-time flow policies on contact

PolyLayout: multi-room 3D layout estimation

Axis Robotics partners with Boost Robotics

Niantic Builds City Digital Twin for Physical AI

GEN-1 Embodied Foundation Model: GPT-3 Moment for Robotics

UnrealTwin: 36M Splat 3D Snow Simulation on WebGPU

SuperMap: Spatio-Temporal SLAM for Visual-Language Navigation

Neural Harmonic Textures: Real-Time Novel View Synthesis

REMIND Re-ID Tracker for Indoor Navigation

PRISM-VO: Scale-Aware Visual Odometry via Photometric Plenoptic Bundle Adjustment

mimic-video Beats pi0 VLAs on LIBERO

Monocular Camera Autonomous Driving Perception

ImmersiVERSE: Physics-Based LiDAR Simulation on UE + Isaac Sim

Xynova Humanoid Robot Learns to Write by Hand

Open-set object detection recognize unseen objects

Walden Robotics raises 300M at 1.1B valuation

Wi-Fi through-wall human sensing open-source system

VLK: Vision-Language-Kinematics for Humanoid Loco-Manipulation

Dexterous Hand Origami vs Japanese Master

Scaling Behavior Foundation Model for Humanoid Robots

Software Replaces Expensive Robot Sensor Arrays

Hyundai Partners NVIDIA on Robot Reference Platform

Scaling Grasping via Egocentric Human Data

SLAM3R: Real-time 3D Scene from Single Video

Tool pick-and-place: why Physical AI is harder than imagined

Sharpa MoDE-VLA Dual Dexterous Hands Peel Apple Autonomously

Xiaomi Xiaomi-Robotics-1 VLA Scaling Laws

NVIDIA ArtiFixer Repairs Incomplete Gaussian Splats

FlexionAI: RGB-Based RL Policy Training in Simulation

Industrial Robotics: Operating Systems Matter More Than Hardware

T800 robot fights in China — robots being blind is the big issue

ClothTransformer — high-quality cloth simulation dataset (493K frames)

Robot autonomously traverses a room while building a map

An incredible diversity of robots running in the real world, demonstrating real applications. Including end-to-end models

✨🇨🇳At the 2026 World Artificial Intelligence Conference in Shanghai, ...

Papers (54)

WilLaGS: Latent-Conditional 3D Appearance Fields for Robust Gaussian Splatting In-the-Wild

LAC: Linear and Angular Compliance for Humanoid Whole-body Control

SOLO: Stable Omni-terrain Long-Horizon Perceptive Humanoid Locomotion

DELTA: Deformable Elevation-Based Local Terrain Attention Encoder for Sparse-Terrain Quadrupedal Locomotion

GigaBrain-WBC-0.5: A Behavior World Model for Robust Whole-Body Control with Environment Interaction

Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data

Pre-training Visual Dexterity in Simulation

DL-SLAM: Enabling High-Fidelity Gaussian Splatting SLAM in Dynamic Environments based on Dual-Level Probability

Learning Agile Navigation in Crowded Environments for Quadruped Robots

HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning

VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes

PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud

TRACE: Ergodic Trajectory Optimization for Active Scene Reconstruction

StreamSplat: Streaming Feed-Forward 3D Gaussian Splatting

PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball

Learning Humanoid Standing-up Control across Diverse Postures

A Framework for Deploying Learning-based Quadruped Loco-Manipulation

Learning Terrain-Aware Whole-Body Control for Perceptive Legged Loco-Manipulation

Autonomous Grasping On Quadruped Robot With Task Level Interaction

$N_0$-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens

Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping

Self-Supervised Consistency Enhanced Disentangled Learning for Neural Decoding Generalization in Brain-Machine Interface

Moving-Horizon Estimation and Nonlinear Model Predictive Control of Cable-Driven Soft Manipulators

Effective Parameters, Real Behavior: Renormalization for Robotics -- From Infinite Electron Mass to Sim-to-Real Gap

Learning Adaptive Multi-Task Guidance, Navigation, and Control via Hypernetworks

KAI: A Kinematic-Aware Interface for Data-Efficient Articulated Object Manipulation