TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model

Paper Metadata

Item	Details
Title	TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model
Authors	Anqi Li, Yuxin Chen, Zhaobo Li, Zhuo Cao (equal contribution), Junli Ren (project lead), Masayoshi Tomizuka, Dhruv Shah (corresponding author)
Affiliations	University of California, Berkeley; Peking University; Tsinghua University; The University of Hong Kong; Princeton University
arXiv	2609.09158v1 (2026-09-08, cs.RO), accepted to CoRL 2026
Project page	tango-vla.github.io/tango-vla.github.io; the paper commits to open-sourcing the data pipeline, generated dataset, VLA framework, model checkpoint, and deployment system, though no code link appears on the page yet
Hardware	Unitree G1; RealSense D455 (front) + D435i (downward), RGB only; the VLA runs on a cluster server with an RTX PRO 6000 while the SONIC tracker runs on an onboard Jetson Orin NX, connected over standard IP networking
Model and training	Qwen2.5VL-7B (warm-started from InternVLA-N1 weights) + a flow-matching MM-DiT action expert; 16 nodes × 8× A100 for about 7 hours, totaling 896 A100 GPU-hours; a single epoch at lr $1\times 10^{-5}$
Dataset	64,633 trajectories synthesized by the PET pipeline; scenes from VLNVerse (263→205) and SAGE-3D (1000→373), 578 augmented scenes in total; PET 86 + rendering 125 = 211 RTX PRO 6000 GPU-hours

One-Sentence Summary

TANGO reframes navigation as a whole-body problem: instead of emitting 2D waypoints or planar velocity commands, a Qwen2.5VL-7B backbone plus a flow-matching action expert directly predicts action chunks of 29-DoF joint angles and a 6D base rotation that a SONIC tracker executes, and a companion Plan–Edit–Track (PET) pipeline automatically synthesizes 64,633 collision-free whole-body traversal trajectories for just 211 GPU-hours, yielding 54.69/52.89 seen/unseen success on VLNVerse and cutting collision rate from 15.81% to 9.90% in cluttered scenes — with RGB only, while its HumanoidPF-based competitor also had LiDAR.

Figure 1: Paper Fig. 1. TANGO as a whole-body foundation model for cluttered indoor navigation, completing a 30-meter route zero-shot in a real-world cluttered office scene and traversing clutter with overhead obstacles, floor obstacles, and narrow passages.

Background and Motivation

The authors open with a clear position: navigation is a whole-body problem. Wheeled or mobile-base robots can reduce navigation to planar path planning because their geometry stays essentially fixed during motion; a humanoid navigates with a high-dimensional articulated body whose geometry changes continuously as it moves, which makes the robot's body configuration an inseparable part of the navigation problem. In cluttered indoor spaces, whether a route is traversable depends not only on the intended path but also on whether the arms, torso, and legs can physically move through the surrounding geometry without collision.

This creates tight coupling: an action that looks valid at the planning level may be infeasible for the embodied humanoid to execute. The authors call this the embodied gap. Existing work splits into three lines, each missing a different piece:

Line of work	Representatives	Strength	What is missing
Classical VLN	Seq2Seq, CMA, RDP, HNR, Uni-NaVid, InternVLA-N1	Language and scene understanding, long-horizon navigation	Action spaces are 2D waypoints or discrete commands, so they cannot represent the relationship between navigation decisions and whole-body feasibility, and fail in spatially constrained traversal
Humanoid foundation models / whole-body VLA	GR00T-N1.6, $\Psi_{0}$, WholeBodyVLA, LeVERB, HumanoidVLA, PhysiFlow	Strong whole-body control, loco-manipulation	Navigation is still represented as high-level locomotion commands delegated to a downstream controller, so whole-body traversability cannot be reasoned about explicitly during navigation; decoupled designs (upper-body motion plus lower-body commands) also limit the whole-body coordination that cluttered traversal requires
Collision-aware traversal (RL)	HumanoidPF, humanoid parkour family	High success in specific traversal scenarios, real avoidance motions	Relies on task-specific priors and training distributions, so it is hard to scale to long-horizon language-guided navigation and complex obstacle compositions; parkour-style work is mostly short-horizon interaction with scene objects

Because the three lines never converge, jointly reasoning about navigation intent and whole-body traversability remains an open gap. TANGO's answer is to change the action space itself: given a language instruction and egocentric RGB observations, it directly predicts 29-DoF joint-space actions, removing the need for separate navigation and control modules.

Changing the action space immediately raises a second problem: where does the training data come from. Covering both semantic task diversity and physically plausible whole-body traversal behavior is not achievable through human motion capture plus retargeting, which introduces motion degradation and embodiment mismatch and does not scale. So the other half of the paper's contribution is PET, a scalable simulation pipeline that automatically synthesizes collision-free humanoid traversal trajectories with dynamically feasible supervision. At deployment, a robust motion tracker with real-time action chunking executes the learned policy for zero-shot transfer to real hardware. The authors explicitly commit to open-sourcing the data pipeline, generated dataset, VLA framework, model checkpoint, and deployment system.

Figure 2: Paper Fig. 2. Top: the PET (Plan–Edit–Track) pipeline automatically synthesizing collision-free whole-body traversal data. Bottom: the TANGO model, combining a vision-language backbone, a diffusion-based action expert, and a low-level tracker for language-guided whole-body humanoid control.

Preliminaries: Formalizing Whole-Body Vision-Language Navigation

Existing VLN work is fundamentally limited by planar action spaces that cannot represent complex traversing motions in real environments. The authors therefore define whole-body vision-language navigation: given a natural language instruction $\ell$ and a current observation $\mathbf{o}_{t}$ containing a temporal sequence of RGB images from front and downward cameras $\mathbf{I}_{1:t}^{\text{fr},\text{dn}}$ plus the whole-body joint-angle proprioceptive state $\mathbf{q}_{t}$, the model learns to predict a whole-body action chunk over a horizon $H$:

$$\mathbf{A}_{t}=\{\mathbf{a}_{1},\cdots,\mathbf{a}_{H}\},\qquad \mathbf{a}_{i}=\{\mathbf{q}_{\text{d},i},\mathbf{r}_{\text{b},i}\},\quad i\in\{1,\cdots,H\}$$

where $\mathbf{q}_{\text{d},i}\in\mathbb{R}^{29}$ is the desired whole-body joint angle at step $i$ and $\mathbf{r}_{\text{b},i}\in\mathbb{R}^{6}$ is the base 6D rotation representation. The predicted chunk is then streamed to a low-level motion tracker for physically grounded navigation.

One design choice here deserves attention: the action is not an end-effector pose or a planar velocity, but joint angles plus base rotation. The model is effectively required to generate "a whole-body reference that a tracker can execute", which pushes traversability reasoning into action generation itself.

Method

1. Environment Augmentation: Turning Standard Rooms into Scenes That Require Whole-Body Motion

Existing navigation datasets (VLNVerse, SAGE-3D, the Habitat family) mainly capture standard room layouts, whereas real environments are characterized by randomly placed objects creating complex spatial constraints. The authors start from VLNVerse (263 scenes) and SAGE-3D (1,000 scenes), filter out low-quality scenes with Gemini 2.5 Flash (uneven ground, incomplete geometry), and keep 205 + 373 = 578 source scenes for augmentation.

They follow HumanoidPF's three obstacle categories, but the decisive difference is placement: obstacles are not scattered randomly but placed trajectory-conditioned. Given an original scene, its A* navigation trajectories, and the occupancy map extracted from the scene, obstacles are inserted along paths exactly where whole-body behaviors are likely to be required. The three targeted interaction types are:

Type	Placement	Induced behavior	Assets used	Share
stride	A low obstacle placed on the path	Stepping over	Rugs, cushions, stools	44%
sidle	A pair of side obstacles forming a narrow passage	Sidling through	Chairs, plants, shelves	15%
squat	An overhead or torso-height obstacle	Ducking or lowering the body	Ceiling lights, lamps, curtains, wall-mounted objects	41%

Candidate placements per scene are sampled along valid A* trajectories while avoiding start and goal regions, and each candidate is aligned with the local path direction so the inserted obstacle naturally interacts with the robot's intended route. Placement also passes validity checks: obstacles must not block unrelated paths or sit too close to trajectory endpoints, though the source trajectory itself is allowed to be affected — that is the whole point. After placement, augmented obstacles are written back into the occupancy representation so downstream planners observe the same geometry as the policy. Obstacle poses are randomly sampled within traversable ranges rather than fixed per category.

Figure 3: Two examples from paper Fig. 5, targeted environment augmentation. Top: a stride case with a low obstacle placed on the path to induce stepping over. Bottom: a squat case with an overhead/torso-height obstacle to induce ducking. The original figure also includes the sidle case (side obstacles forming a narrow passage).

2. The PET Pipeline: Plan — Edit — Track

Humanoid motion collection usually depends on labor-intensive motion capture plus retargeting, which introduces motion degradation and embodiment mismatch. PET is a fully automatic offline pipeline that synthesizes collision-free, dynamically feasible whole-body traversal motions. The three stages have distinct jobs, and the way responsibilities are divided is the most interesting part of the method.

Plan: an obstacle-aware planar reference path. Given start $s$, goal $g$, and the scene occupancy map (which already includes the augmented obstacles), a planar path is planned with A* on the 2D floor grid. To favor safer routes with whole-body clearance, an obstacle-aware term is added to the step cost, computed from the 2D signed distance to occupied cells $\Phi_{2\mathrm{D}}(\mathbf{x})$:

$$c(\mathbf{x})=c_{\text{step}}+\lambda\,\exp\!\left(-\Phi_{2\mathrm{D}}(\mathbf{x})/d_{0}\right)$$

Cells near obstacles receive a soft penalty decaying with clearance $d_{0}$, while the admissible Euclidean-to-goal heuristic preserves optimality under this cost. The output is a planar polyline $\mathcal{P}=\{(x_{k},y_{k})\}_{k=1}^{K}$ with tangent heading $\psi_{k}=\operatorname{atan2}(y_{k+1}-y_{k},x_{k+1}-x_{k})$.

A rule-based yaw rewriter then handles narrow passages: the lateral free width $w_{\perp}(s)$ is probed on both sides of the path along $\mathcal{P}$, and when it falls below a threshold $w_{\min}$, the body heading is rewritten to be offset $90^{\circ}$ from the travel tangent ($\psi^{\text{body}}_{k}=\psi_{k}\pm\tfrac{\pi}{2}$, with the sign chosen so the leading shoulder stays on the wider side). Smooth $\pm 90^{\circ}$ transitions into and out of the passage produce a crab-walking gait in which translation direction and facing direction are decoupled. Finally the trajectory is converted into velocity commands for the SONIC motion planner to produce natural humanoid walking along the planned 2D path; the trajectories are replayed in IsaacSim to render egocentric RGB at 2 Hz, which Gemini 2.5 Flash turns into formatted VLN instructions.

Edit: turning walking motion into whole-body avoidance motion. This stage takes the Plan output $\mathcal{M}^{\text{ref}}$ and produces $\mathcal{M}^{\text{edit}}$ with whole-body obstacle interactions. The reference gait is kept as a locomotion prior — its timing, gait phase, and motion style are preserved, and edits are applied locally only where the path corridor contains obstacles. Scene voxel occupancy is converted into a 3D signed distance field $\Phi(\mathbf{p})$ using the fast marching method, providing distance values and gradients $\nabla\Phi$ throughout the workspace; thin structural obstacles such as bars are additionally represented as exact oriented boxes with an analytic SDF so clearance queries stay continuous. Editing is restricted to a spatial corridor — a tube of half-width $w_{\text{corr}}\approx 0.5\,\mathrm{m}$ around the planned path — and every field sample whose horizontal position lies outside the tube is treated as free space.

At each frame a whole-body inverse-kinematics (IK) problem is solved with four objectives: reference posture tracking, foot contact and landing targets, a center-of-mass (CoM) target, and potential-field (PF) link forces. The PF forces come from the HumanoidPF humanoid potential field, guided by obstacle and ground-truth trajectory priors, applied to key body links through SoftMimic-style pseudo-forces. The three behavior families are handled by different mechanisms:

Obstacle type	Mechanism	Key details
squat	Potential-field crouch controller	For squat obstacles only the vertical component of each force is kept, so the field provides a body-lowering cue rather than a lateral push. Two extra terms are added: a look-ahead term (probing $\tilde{\Phi}$ ahead of the upper body at several head and shoulder heights and forward distances, and adding a downward force proportional to the violation when the minimum probe lies within a margin $\eta$ of the geometry, so the body begins to lower before reaching the obstacle) and a virtual head barrier (protecting the point $\mathbf{p}_{\text{head}}=\mathbf{p}_{\text{torso}}+h\hat{z}$ with clearance $b=\tilde{\Phi}(\mathbf{p}_{\text{head}})-r$; when $b<b_{m}$ it applies a strictly downward force $k_{s}(b_{m}-b)+k_{h}\max(0,-b)$, i.e. soft and hard barrier components, clipped to a maximum trunk displacement). The resulting downward upper-body displacement is converted into a coordinated squat by coupling the crouch activation $\alpha=\operatorname{clip}(\Delta z/\delta_{\max},0,1)$ to posture: a forward waist-pitch target $\theta^{\text{waist}}\leftarrow\theta^{\text{waist}}+\alpha\theta_{0}$, a proportional hip-pitch bias, and a downward CoM target shift $\Delta z_{\text{CoM}}=\alpha c_{z}$ — this CoM shift lets the legs lower the body instead of letting the CoM task pull it back upright. Three coordinated presets are implemented: waist-only (bend at the waist while the tracker controls the legs), com-drop (lower the body vertically), and full-squat (combining waist motion, pelvis tilt, hip flexion, knee bending, ankle dorsiflexion, and CoM lowering). During an active crouch the upper-body reference-tracking terms are relaxed so the field can reshape the trunk instead of competing with the upright reference
stride	Gait-adaptation module	Upper-body guidance is insufficient because the swing feet must actually step over the object. The module retargets footsteps online while preserving the phase and timing of the reference gait. At each foot lift-off the base is projected onto the planned path, a short arc-length look-ahead is queried, and the landing target is specified in the path-yaw frame, clamped to a maximum displacement from the reference touchdown (preventing the IK solver from pursuing unreachable foot placements). A footprint-aware SDF scan detects a low obstacle ahead of the swing foot and returns the interval $[d_{\text{near}},d_{\text{far}}]$ spanned by the obstacle plus its top height $z_{\text{top}}$; obstacles are classified by height, and only $z_{\text{top}}\leq z_{\text{bar}}$ (a steppable bar) triggers a step-over motion, while taller geometry is treated as a wall or ceiling and handled by the lateral and upper-body layers, so the robot never tries to step over a wall
sidle	Crab-walking from the Plan stage plus lateral PF guidance	Horizontal components push the trunk and arms away from the side walls, coordinated with the crab-walking heading

For stride obstacles the landing target is pushed past the far edge by a margin: $d^{\text{land}}=d_{\text{far}}+m$. The landing objective is evaluated against this target rather than the reference touchdown on the bar, which both clears the obstacle and compensates for the known forward-reach undershoot of the downstream tracker, keeping the realized landing past the bar. The swing trajectory is formed by warping the reference foot path (from motion capture or RL data) onto the retargeted endpoints and adding a vertical clearance arc: with swing phase $u\in[0,1]$, smoothstep-interpolated $\bar{u}$, and reference foot path $\mathbf{p}^{\text{ref}}(u)$,

$$\mathbf{p}^{\text{swing}}(u)=\mathbf{p}^{\text{ref}}(u)+(1-\bar{u})\,\Delta\mathbf{p}_{\text{lift}}+\bar{u}\,\Delta\mathbf{p}_{\text{land}}+h_{\text{arc}}\,\rho(u)\,\hat{z}$$

where $\Delta\mathbf{p}_{\text{lift}},\Delta\mathbf{p}_{\text{land}}$ are the offsets between retargeted and reference endpoints, and $\rho(u)$ is an asymmetric clearance profile with a fast quarter-sine rise to an early peak followed by a quarter-cosine fall. The asymmetry is deliberate — it keeps the foot high early in the swing, which the trailing leg crossing the bar shortly after lift-off requires; a symmetric $\sin(\pi u)$ arc would be too low at that stage. The peak height $h_{\text{arc}}$ is solved per step so the full foot footprint clears the obstacle box at every sampled phase:

$$z_{\text{ground}}+h_{\text{arc}}\,\rho(u)\ \geq\ z_{\text{top}}(u)+\epsilon$$

The solver and the executed arc use the same profile $\rho$, keeping planned and executed motion consistent. A swing-knee bend bias encourages knee flexion rather than straight-leg extension for clearance; when several low obstacles appear in sequence, a short foothold sequence places natural-length steps through the gaps instead of demanding one long step.

Track: the SONIC tracker as an executability filter. The edited motion $\mathcal{M}^{\text{edit}}$ is kinematic and may violate dynamic feasibility, for example through overly fast swings or aggressive crouches. Feasibility is restored by passing it through the SONIC tracker in closed loop: the edited whole-body reference (joint positions, joint velocities, base orientation) is streamed frame by frame to the RL tracking policy, which tracks it under physics in MuJoCo and outputs a dynamically feasible motion $\mathcal{M}^{\text{track}}$. Failing or colliding trajectories are discarded outright. The same tracker is used at deployment (system-0), so synthesized supervision is consistent with execution.

Here lies a counterintuitive but crucial decision: the training supervision is not the tracked trajectory. The authors state explicitly that using tracked trajectories as supervision might degrade the human-like quality of the motion, so they instead use the collision-free reference motions from the planning and editing stages as the action supervision signal. This preserves human-like structure while still ensuring physical executability, because the Track stage has already filtered out whatever was not executable. Based on the verified trajectories, RGB observations are regenerated and language instructions updated accordingly.

The final dataset contains 64,633 trajectories; PET and rendering require 86 and 125 RTX PRO 6000 GPU-hours respectively, for a total of 211 GPU-hours (paper Table 6). For a dataset of that scale in whole-body navigation, this cost is remarkably low.

Two further training-side measures target rare behaviors. After converting rollouts into action chunks, data-level rebalancing upsamples behaviorally important rows (large turns, sideways motion, squat, stride, and mixed squat-stride segments), with stride and sideways motion boosted most strongly to compensate for their lower frequency and weaker action signal; loss-level weighting applies motion-conditioned per-dimension weights on the action vector, assigning higher weights to the joint-position dimensions most associated with each behavior (hip, spine, and knee for squat; spine and shoulder for stride; their union for mixed motions; shoulder plus body-frame motion dimensions for sideways motion).

flowchart TD
  subgraph DATA["PET data pipeline (offline, 211 RTX PRO 6000 GPU-h)"]
    SC["VLNVerse 263 + SAGE-3D 1000
Gemini 2.5 Flash filter -> 578 scenes"] --> AUG["Trajectory-conditioned augmentation
stride 44% / sidle 15% / squat 41%"]
    AUG --> PLAN["Plan: A* with obstacle-aware cost
c(x) = c_step + lambda*exp(-Phi_2D/d_0)
+ 90-deg yaw rewriter -> crab walk"]
    PLAN --> SONICP["SONIC motion planner
-> natural walking M_ref"]
    SONICP --> EDIT["Edit: SoftMimic pseudo-forces
whole-body IK, corridor half-width 0.5 m
SDF via fast marching"]
    EDIT --> MEDIT["M_edit: arm clearance,
step-over, crouch"]
    MEDIT --> TRACK["Track: SONIC as executability filter
MuJoCo physics, discard collisions"]
    TRACK --> SUP["Supervision = collision-free REFERENCE
(not tracked trajectory)"]
    SUP --> DS["64,633 trajectories
+ 2 Hz RGB replay + Gemini instructions"]
  end
  subgraph MODEL["TANGO triple-system model (896 A100-h)"]
    IMG["Front + down RGB, vertically stacked"] --> BATS["BATS token sampling
P(t)=(1-eps)e^(k(t-T)/T)+eps"]
    BATS --> GP["Grid pooling: fine for recent,
coarse for history"]
    GP --> VLM["System-2: Qwen2.5VL-7B
warm-started from InternVLA-N1
-> latent z"]
    LANG["Language instruction l"] --> VLM
    VLM --> CE["Text-decoding branch
VideoQA co-tuning (L_CE)"]
    VLM --> S1["System-1: flow-matching MM-DiT
stabilized target a_tilde with
chunk-relative yaw + dx,dy,dpsi"]
    PROP["Proprioception q_d,t"] --> S1
    S1 --> RTC["Training-time RTC
random committed prefix d
inpaint remaining horizon"]
    RTC --> FM["L_FM, weight w_FM = 20"]
  end
  DS --> S1
  CE --> LOSS["L = L_CE + w_FM * L_FM
1 epoch, lr 1e-5"]
  FM --> LOSS
  S1 --> S0["System-0: SONIC tracker
~200 Hz joint commands"]
  S0 --> G1["Unitree G1 zero-shot"]

3. The TANGO Architecture: Three Systems

Whole-body VLA navigation in complex environments simultaneously demands physical world understanding, continuous action prediction, and robust action execution. TANGO therefore adopts a triple-system architecture: a vision-language backbone (system-2), an MM-DiT action expert with real-time chunking (system-1), and an off-the-shelf motion tracker (system-0). The VL backbone and the action expert are trained jointly; at deployment the predicted action chunks are streamed to the low-level tracker to generate continuous high-frequency humanoid control signals.

System-2: vision-language perception. Instantiated with Qwen2.5VL-7B and warm-started from InternVLA-N1 weights to inherit strong navigation priors (chosen for open-source availability and navigation pretraining; other VLA-based VLN backbones could also be adapted to this framework). At each timestep the front and downward camera views $\mathbf{I}_{t}^{\text{fr}},\mathbf{I}_{t}^{\text{dn}}$ are vertically stacked into a single frame $\mathbf{I}_{t}$. To manage long-horizon video history $\mathbf{I}_{1:T}$ within a given memory capacity, Budget-Aware Token Sampling (BATS) is applied: at timestep $T$, all history frames are sampled independently into the navigation context according to

$$P(t)=(1-\epsilon)\,e^{k(t-T)/T}+\epsilon,\qquad t\in[1,T]$$

where $\epsilon$ and $k$ regulate temporal intensity. This is an exponentially decaying sampling distribution with a floor at $\epsilon$, meaning recent frames are almost always sampled while distant frames retain a nonzero chance of being kept — appropriate for long-horizon navigation, where information like "which room did I just leave" cannot be discarded entirely. Visual features $v_{i}=\text{VisionEncoder}(\mathbf{I}_{i})\in\mathbb{R}^{n\times p\times c}$ are further spatial-grid pooled via $\tilde{v}_{i}=\mathrm{GridPool}(v_{i},g_{i})\in\mathbb{R}^{g_{i}\times c}$, allocating finer grids to recent observations and coarser grids to history. Finally the sampled visual tokens $\tilde{v}_{\text{sampled}}$ and the language instruction $\ell$ are fed into the VLM to produce a latent context token $z=\text{VLM}(\tilde{v}_{\text{sampled}},\ell)$.

System-1: action prediction. Conditioned on the latent $z$ and current proprioception $\mathbf{q}_{d,t}$, system-1 predicts a future whole-body reference chunk. Although standard actions are defined as joint angles and base poses $\mathbf{a}_{i}=\{\mathbf{q}_{d,i},\mathbf{r}_{b,i}\}$, the authors formulate a stabilized training target to facilitate regression:

$$\tilde{\mathbf{a}}_{i}=\{\mathbf{q}_{d,i},\tilde{\mathbf{r}}_{b,i},\Delta x_{i},\Delta y_{i},\Delta\psi_{i}\}$$

where the base yaw in $\tilde{\mathbf{r}}_{b,i}$ is parameterized relative to the first frame of the chunk (avoiding the regression difficulty of absolute heading), and the auxiliary deltas $(\Delta x_{i},\Delta y_{i},\Delta\psi_{i})$ explicitly encode chunk-level planar displacement and heading change — planar trajectory prediction is thus an auxiliary objective rather than the main one, a fact later exploited in the Table 4 ablation to isolate the contribution of whole-body action generation. System-1 is implemented as a flow-based MM-DiT trained via flow matching to generate the horizon $\tilde{\mathbf{A}}_{t:t+H}$.

To align offline training with online streaming execution, training-time RTC (real-time chunking) is applied: the model is conditioned on a randomized committed prefix of $d$ actions and must inpaint the remaining horizon. This mirrors the deployment reality directly — the first few actions of the previous chunk are already executing and cannot be retracted, so the model must continue from them continuously. The generated chunk is then recovered to $\mathbf{A}_{t:t+H}$ and streamed to the SONIC tracker (system-0), which tracks the reference against robot proprioception to provide high-frequency joint commands.

Joint training objective. Following dual-branch designs for VL decoding, a text-decoding branch is appended to system-2, and navigation tasks are co-tuned alongside VideoQA samples to preserve generalized world knowledge:

$$\mathcal{L}=\mathcal{L}_{\text{CE}}+w_{\text{FM}}\cdot\mathcal{L}_{\text{FM}},\qquad w_{\text{FM}}=20$$

where $\mathcal{L}_{\text{CE}}$ is the cross-entropy loss for VideoQA and $\mathcal{L}_{\text{FM}}$ the flow-matching loss. TANGO is trained end-to-end for a single epoch at learning rate $1\times 10^{-5}$, with full-parameter tuning on 16 nodes of 8× NVIDIA A100 GPUs for approximately 7 hours, totaling 896 A100 GPU-hours.

4. Deployment: Cloud-Edge Separation

Figure 4: Paper Fig. 3, the real-world deployment system. A server-client design runs the low-frequency VLA model and the high-frequency whole-body tracker separately.

The deployment architecture separates the compute-intensive VLA module from the high-frequency whole-body control module: the VLA (system-2 and system-1) runs on a cluster server equipped with an RTX PRO 6000, while the SONIC tracker (system-0) runs on an onboard Jetson Orin NX, the two connected through standard IP networking. The humanoid captures front- and downward-facing RGB with RealSense D455 and D435i cameras and streams it, together with proprioceptive states, to the server with approximately 20 ms latency. The server acts as a global clock and performs VLA inference every 0.5 s, matching an execution horizon of $s=15$ actions at 30 Hz. Predicted motion chunks are streamed back to the robot, resampled to 50 Hz, and executed by the SONIC tracker, which closes the low-level control loop at approximately 200 Hz. The authors also built a webpage-based control panel for sending navigation instructions, monitoring VLA output and robot observation, and sending control signals to the VLA and WBC systems.

Simulation deployment uses a different trick. Whole-body navigation demands both high rendering quality and physical authenticity, which a single simulator struggles to provide. The authors use a digital twin teleportation system: MuJoCo handles low-level tracker deployment and physical simulation, while a humanoid digital twin is teleported in IsaacSim, from which photo-realistic visual observations are acquired at a designated camera pose solved by forward kinematics (FK) from a given humanoid robot pose and used as VLA input.

Experiments

1. VLNVerse Benchmark: The Only Method with Low-Level Physical Control

Evaluation is on VLNVerse, a newly established VLN benchmark with photorealistic indoor scenes in IsaacSim, comparing against discrete-action models CMA and Seq2Seq, the continuous-action model RDP, the neural implicit representation method HNR, and state-of-the-art VLA models InternVLA-N1 and Uni-NaVid. Metrics are Success Rate (SR), Success weighted by Path Length (SPL), Navigation Error (NE), and Oracle Success Rate (OSR). All methods are trained on 3,963 trajectories from VLNVerse-train and evaluated on VLNVerse-seen (423) and VLNVerse-unseen (825); InternVLA-N1 and Uni-NaVid are fine-tuned for five epochs following their original training settings.

One prerequisite must be stated clearly: all baselines lack low-level control modules and are therefore evaluated in a teleportation setting following the original VLNVerse protocol, while TANGO is the only method equipped with low-level control, operating under realistic physical constraints.

Method	Low-level control	Seen NE ↓	Seen OSR ↑	Seen SR ↑	Seen SPL ↑	Unseen NE ↓	Unseen OSR ↑	Unseen SR ↑	Unseen SPL ↑
CMA	✗	5.36	59.81	37.35	33.36	5.16	62.79	31.15	27.92
Seq2Seq	✗	4.78	44.68	32.62	30.39	4.36	49.58	35.03	33.37
RDP	✗	4.02	68.09	47.28	41.69	3.75	71.93	48.60	42.72
HNR	✗	–	–	36.34	32.10	–	–	32.95	29.56
InternVLA-N1	✗	3.91	62.50	51.56	34.37	4.09	64.13	45.56	34.98
Uni-NaVid	✗	4.31	60.62	51.88	39.72	3.97	62.50	45.00	39.42
TANGO (ours)	✓	3.90	70.31	54.69	40.18	3.72	71.07	52.89	40.18

TANGO achieves the highest SR and lowest reported NE on both VLNVerse-seen and VLNVerse-unseen, with SPL and OSR comparable to the state-of-the-art RDP baseline. Given that RDP is evaluated under teleportation — exempt from every difficulty of physical execution — this comparison favors TANGO.

The authors offer a candid explanation for the lower SPL: relatively lower SPL suggests reduced navigation efficiency, likely due to the conservative behavior of the low-level tracker, which may favor safer but less direct trajectories around obstacles. This is an honest statement of trade-off: path optimality given up in exchange for physical feasibility.

2. Cluttered Traversal: RGB Beats a Competitor Equipped with LiDAR

Evaluation is on augmented VLNVerse-unseen scenes, verifying obstacle-avoidance and spatial understanding in 3D environments. Two low-level executors are compared: the official Unitree RL controller with an MPC module (executing basic movement from planar velocity commands), and the HumanoidPF generalist policy (performing obstacle-avoidance motions in complex 3D scenes). For the fine-tuned setting, InternVLA-N1 is fine-tuned in augmented VLNVerse-train scenes with visual input collected from either Unitree-controller or HumanoidPF motions (denoted "+Unitree WBC" and "+HumanoidPF"). At inference the Unitree controller takes 2D velocity commands interpreted from the predicted trajectory, while HumanoidPF tracks waypoints sampled 1.2 m ahead on the trajectory, both at 50 Hz. The authors introduce Collision Rate (CR), the percentage of evaluation episodes with at least one collision, to better quantify navigation safety in clutter.

Method	NE ↓	SR ↑	SPL ↑	CR ↓
InternVLA-N1 (zero-shot) †	5.34	26.67	11.92	19.03
InternVLA-N1 (zero-shot) ‡	3.19	35.29	24.35	16.11
InternVLA-N1 + Unitree WBC	4.57	35.00	17.38	17.49
InternVLA-N1 + HumanoidPF	4.04	41.88	29.49	15.81
TANGO (ours)	4.01	43.75	31.83	9.90

† Equipped with the Unitree low-level controller and MPC; ‡ tracked by the HumanoidPF generalist policy.

TANGO achieves the highest SR and SPL while maintaining the lowest CR across all methods. The most important number is collision rate: a reduction from 15.81% to 9.90% versus the strongest baseline, even though TANGO relies solely on RGB observations while the HumanoidPF-tracked baseline additionally has access to LiDAR-based geometric perception. Success rate improves by 1.87 percentage points and SPL by 2.34 points over the fine-tuned InternVLA-N1 + HumanoidPF baseline. The authors conclude that this supports the benefit of end-to-end whole-body action generation over the evaluated modular approaches.

This comparison merits a further word. The chronic weakness of modular schemes is interface loss: VLA emits a trajectory, which is interpreted into velocity commands or waypoints, which a controller then executes — some geometric information is lost at each crossing, and "the trajectory is feasible" does not imply "the body can get through". TANGO pushes traversability reasoning into action generation itself, which is why it achieves a lower collision rate with equal or weaker sensing.

3. Real-World Experiments: Zero-Shot Sim-to-Real

Figure 5: Paper Fig. 4, real-world qualitative results. Four scenarios: long-horizon navigation, side-stepping through a narrow pathway, bending down to avoid overhead obstacles, and stepping over obstacles on the ground — demonstrating zero-shot sim-to-real transfer without any real-world training data.

Qualitative. Four representative scenarios: long-horizon navigation, side-stepping through a narrow pathway, bending down to avoid overhead obstacles, and stepping over ground obstacles. TANGO demonstrates robust scene understanding and spatial traversal capability in all cases: the robot executes continuous whole-body motions while following natural-language instructions, adapts its body configuration to negotiate obstacles, and maintains progress toward the navigation goal.

Quantitative. Comparison against the fine-tuned InternVLA-N1 + Unitree WBC baseline in three settings: short-horizon 2D navigation (one turn, approximately 10 m), long-horizon 2D navigation (two or three turns, approximately 30 m), and cluttered 3D navigation with one challenging obstacle. Each setting contains three scenes with five trials per scene, giving 15 trials per method per setting. Cluttered-scene instructions explicitly specify the required traversal behavior (stepping over, side-stepping, or bending down). The table below reports successful trials and mean collisions per trial (the latter is a collision count, distinct from the episode-level CR used in simulation):

Method	Short-horizon 2D SR ↑	Short-horizon Coll. ↓	Long-horizon 2D SR ↑	Long-horizon Coll. ↓	Cluttered 3D SR ↑	Cluttered 3D Coll. ↓
InternVLA-N1 †	11/15	1.40	6/15	3.47	6/15	1.93
TANGO (ours)	12/15	0.40	8/15	1.07	10/15	0.73

TANGO scores the highest SR across all three tested settings with fewer collisions than the baseline. The largest gap is in the cluttered 3D setting: 10/15 versus 6/15, with mean collisions of 0.73 versus 1.93 — precisely where the whole-body action space should matter most. Long-horizon 2D navigation (about 30 m) improves from 6/15 to 8/15, indicating that error accumulation over long horizons is also suppressed.

4. Ablation One: What the Whole-Body Action Space Is Actually Worth

The impact of action representation and low-level execution is studied on the original VLNVerse-unseen benchmark to validate the 29-DoF whole-body action space. Comparisons are against zero-shot InternVLA-N1 and Ours-2D, a planar variant of the proposed method. Since planar trajectory prediction is only an auxiliary objective in the formulation, Ours-2D cleanly isolates the effect of whole-body action generation. Both baselines are evaluated under two execution settings: teleportation, and physical execution using the Unitree low-level controller with MPC.

Method	Low-level control	Val Unseen SR ↑	Val Unseen SPL ↑
InternVLA-N1	✗ (teleportation)	43.69	35.74
Ours-2D	✗ (teleportation)	45.74	35.44
InternVLA-N1 †	✓ (physical execution)	42.37	22.50
Ours-2D †	✓ (physical execution)	26.67	8.34
TANGO (whole-body)	✓ (physical execution)	52.89	40.18

† Equipped with the Unitree low-level controller and MPC.

This table carries the strongest argumentative weight in the paper because it quantifies the teleportation-to-physical-execution gap:

First, both planar policies drop substantially when moving from teleportation to physical execution. Ours-2D falls from 45.74% to 26.67% (−19.07 points) and its SPL from 35.44 to 8.34 (−27.10 points); InternVLA-N1 falls from 43.69% to 42.37% with SPL from 35.74 to 22.50. This directly confirms the paper's central claim about how hard conventional navigation policies are to transfer to embodied settings — a model that looks quite capable under teleportation can collapse once it must actually walk its body through.

Second, TANGO is higher under physical execution (52.89%), not merely avoiding a drop but exceeding both baselines under the more forgiving teleportation setting. The Ours-2D versus TANGO comparison (26.67% vs 52.89%) supports the benefit of whole-body action prediction: the only difference is the action space, and the planar version, having discarded whole-body information, is exposed immediately under physical execution.

Note the authors' own caveat: the zero-shot InternVLA-N1 rows provide additional reference points but do not isolate action representation from training differences — the clean comparison is Ours-2D versus TANGO, which share the training recipe.

5. Ablation Two: RTC Is the Difference Between Working and Not

Motion editing, RTC, and the low-level tracker are each evaluated on augmented VLNVerse-unseen (the default configuration uses SONIC; the tracker variant uses ScaleBFM; SR and CR are percentages and SPL is scaled by 100):

Method	SR ↑	SPL ↑	CR ↓
w/o RTC	10.94	10.94	14.60
w/o Motion Editing	36.25	30.36	20.60
SONIC → ScaleBFM	40.94	29.65	9.10
TANGO (full)	43.75	31.83	9.90

Removing RTC is catastrophic: SR drops from 43.75% to 10.94%, a fall of 32.81 percentage points, with CR rising from 9.90% to 14.60%. A drop of that magnitude shows RTC is not an optimization trick but a precondition for the system working at all, because it maintains motion continuity across action chunks during execution. The reason is not hard to see: at deployment a new chunk arrives every 0.5 s, and if the beginning of the new chunk is discontinuous with the chunk currently executing, the robot experiences an action jump every half second. Whole-body motion cannot possibly be stitched smoothly under those conditions, leaving the tracker to firefight continuously.

Removing motion editing lowers SR to 36.25% and raises CR to 20.60% (the highest collision rate in the table), showing that obstacle-aware motion supervision contributes materially to collision avoidance. The doubling of collision rate is especially telling: without the Edit stage the data contains only ordinary walking motion, so the model never learns the avoidance behaviors — tuck the arms, lower the body, lift the legs.

Replacing SONIC with ScaleBFM yields 40.94% SR, 29.65 SPL, and 9.10% CR, with all three metrics within three points of the default configuration — and CR slightly better. This shows TANGO is compatible with another general whole-body controller, so the method is not bound to a specific tracker, an important scalability signal for deployment across platforms.

Limitations

Author-stated: low-level tracker capability is the key constraint on further deployment. The authors state explicitly in their Limitation paragraph that the capability of the low-level tracker emerges as a key constraint for deployment in more complex environments, for example walking up stairs. This is a structural dependency: the ceiling of TANGO's action space is set by system-0 — the VLA can predict a whole-body reference for stepping up stairs, but if the tracker cannot execute it, the chain breaks there. Notably this paper and tracker-focused work such as PGMT are complementary: PGMT supplies exactly the missing piece, tracker capability on complex terrain.

Author-stated: vision input is RGB only. The model relies solely on RGB images, which may restrict its ability to fully understand complex scenes, particularly in cluttered, visually ambiguous, or low-light environments, where the authors expect depth cameras and LiDAR to help; they leave this to future work. This admission sits in interesting tension with the Section 2 result: precisely because RGB alone beat a LiDAR-equipped baseline, RGB is sufficient for the scenes evaluated here, yet the authors concede it becomes a weakness under low light or visual ambiguity.

Reader-side additions: First, the statistical strength of the real-world quantitative results is limited — only 15 trials per method per setting (3 scenes × 5 trials), so the 10/15 versus 6/15 difference in cluttered 3D carries wide confidence intervals at that sample size, and the paper reports neither significance testing nor cross-scene variance. Second, the simulation comparison conditions are not equivalent: TANGO is the only method evaluated under physical execution while all baselines are evaluated under teleportation. This favors TANGO (it shoulders more difficulty), but it also means the SR/SPL in Table 1 are not measured under the same protocol; a strict comparison requires the "same method, two execution settings" design of Table 4, which covers only Ours-2D and InternVLA-N1. Third, dependence on the cloud-edge network: the VLA runs on a server with inference every 0.5 s and the robot is connected via IP networking with roughly 20 ms latency. The paper does not analyze behavior under network jitter, packet loss, or disconnection — for a humanoid mid-stride over an obstacle, this is not a failure mode that can be ignored. Fourth, the conservatism behind the lower SPL is not analyzed separately: the authors attribute it to a conservative tracker but provide no path-length distribution or quantification of detour magnitude. Fifth, behavior coverage in the synthesized data is determined by rules: the three obstacle categories (stride/sidle/squat) and their proportions (44%/15%/41%) are hand-designed, crouching has only the waist-only/com-drop/full-squat presets, and step-over triggers only for $z_{\text{top}}\leq z_{\text{bar}}$; longer-tail traversal behaviors in the real world (sidling while ducking, stepping over irregular shapes) fall outside coverage. Sixth, code and dataset are not yet released (the paper promises five open-source artifacts, but the project page has no code link yet), so the numbers are not currently reproducible.

Conclusion and Outlook

TANGO's contribution decomposes into three mutually supporting layers. The top layer is a problem restatement: navigation is a whole-body problem, so the action space should be 29-DoF joint angles plus a 6D base rotation rather than 2D waypoints or planar velocities. The restatement is not new in itself, but Table 4 nails it down — planar policies lose 19 points going from teleportation to physical execution (Ours-2D 45.74% → 26.67%), while the whole-body action space reaches 52.89% under physical execution. The middle layer is data: the PET pipeline synthesizes 64,633 collision-free whole-body trajectories for 211 GPU-hours, where the decision to "use the Track stage only as an executability filter and supervise with the collision-free reference from Plan+Edit rather than the tracked trajectory" simultaneously preserves human-like quality and physical feasibility. The bottom layer is engineering: BATS's exponential-in-time sampling plus coarse-to-fine grid pooling keeps long-horizon video history within budget, the stabilized action target (chunk-relative yaw plus auxiliary planar deltas) makes regression trainable, and training-time RTC aligns offline training with online streaming execution — the necessity of RTC being proven beyond argument by that −32.81 point ablation.

On results: 54.69/52.89 seen/unseen SR on VLNVerse as the only method with physical control; CR reduced from 15.81% to 9.90% in augmented cluttered scenes using RGB only against a LiDAR-equipped competitor; zero-shot completion of a 30 m cluttered office route on the real robot, and 10/15 versus 6/15 in the cluttered 3D setting. Swapping in ScaleBFM keeps all three metrics within three points, showing the framework is not bound to a particular tracker.

Looking forward, the authors position TANGO as a step toward practical whole-body large planning models, with future work adapting and scaling it as a foundation model toward more general loco-manipulation foundation models for tasks requiring coordinated, active use of the entire body. Working backward from the constraints this paper exposes, the three most valuable directions are: replacing system-0 with a tracker of stronger terrain capability (adding abilities such as stair climbing, exactly complementary to perceptive-tracker work like PGMT); introducing depth or LiDAR to cover low-light and visually ambiguous scenes; and moving enough compute onboard to run without the server, eliminating the cloud-edge link as a single point of failure.

Golden Quote

For a wheeled robot, navigation asks whether the route is passable. For a humanoid, it asks whether this body can get through. Switching the action space from 2D waypoints to 29 joint angles does not make the problem more complicated — it finally starts asking the question that should have been asked all along.