VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation

Figure 1: Center: a Unitree G1 walking between tables to place and pick objects for 54 loops with the RGB-based sim-to-real policy. Surrounding: diverse simulated scenes used for training.

Background: What Humanoids Are Missing Is Autonomous Loco-Manipulation

Humanoid robots are often framed as the natural embodiment of general-purpose physical intelligence — machines that could ultimately take on a large fraction of physical work for society. Yet despite rapid progress in hardware and control, current humanoids have delivered limited real, sustained productivity outside of carefully engineered demos. A core missing piece is autonomous loco-manipulation — tight coordination of locomotion and manipulation under onboard perception — over long horizons and across diverse environments to accomplish useful tasks.

Most existing humanoid systems either focus on blind locomotion, static tabletop manipulation without mobility, or rely heavily on human teleoperation or non-onboard sensors. There has recently been an exciting push to replicate the large language model recipe in robotics: collect large-scale real-world datasets and train "robotic foundation models" from real-world teleoperation data. But this path is especially expensive for mobile manipulation: mobile settings encounter substantially more variation than fixed tabletop setups and therefore demand far more data. When the mobile platform is a humanoid, the cost per data point increases even further due to hardware complexity, higher degrees of freedom, safety constraints, and the engineering overhead of the teleoperation stack. In other words, if we treat humanoid mobile manipulation as "just another data problem," the required scale may be prohibitively expensive in practice.

Simulation offers an alternative. Modern GPU-accelerated, photorealistic simulators generate orders of magnitude more data at low marginal cost. Sim-to-real has become the de facto approach for legged locomotion, where policies trained in simulation routinely transfer to hardware. Manipulation, in contrast, is still dominated by imitation learning from real-world data, with sim-to-real successes typically restricted to tabletop settings and narrow tasks. Moreover, the two are usually studied in isolation: locomotion work often ignores manipulation, and manipulation work typically assumes a fixed base. VIRAL asks: can visual sim-to-real enable useful autonomous loco-manipulation for humanoids?

Method: A Two-Stage Teacher-Student Framework

Figure 2: The VIRAL teacher-student pipeline. Phase 1: a privileged RL teacher policy receives full-state proprioception and exteroception and outputs WBC commands. Phase 2: a vision-based student observing only RGB images and sim-to-real proprioception is trained to imitate the teacher via DAgger and behavior cloning.

Phase 1: The Privileged RL Teacher (16× L40S)

The teacher is formulated as a goal-conditioned RL policy. At time step $t$, the teacher $\pi_{\text{teacher}}(a_t \mid o^{\text{priv}}_t)$ outputs a high-level command for the low-level whole-body control (WBC) policy given privileged observation:

$$a_t = (\Delta v_t,\ \Delta \omega^{\text{yaw}}_t,\ \Delta q^{\text{arm}}_t,\ \Delta q^{\text{finger}}_t)$$

where $\Delta v_t$ and $\Delta \omega^{\text{yaw}}_t$ are delta linear ($x,y$) and angular (yaw) velocity commands, and $\Delta q^{\text{arm}}_t$, $\Delta q^{\text{finger}}_t$ are delta joint targets for arm and finger motors. The privileged observation $o^{\text{priv}}_t = [o^{\text{prop-priv}}_t,\ o^{\text{exte-priv}}_t]$ contains:

• Privileged proprioception $o^{\text{prop-priv}}_t = [v_t,\ \omega_t,\ g_t,\ a_{t-1},\ q_t,\ \dot{q}_t,\ f^{\text{finger}}_t]$: base linear and angular velocities, base projected gravity, last action, joint positions and velocities, fingertip forces.
• Privileged exteroception $o^{\text{exte-priv}}_t = [e_t,\ T_t,\ O_t]$: current stage $e_t$, placement and lift target $T_t$, and relative transforms of objects and tables to the robot $O_t$.

The teacher is trained with PPO using a custom distributed TRL implementation. This phase runs without the compute burden of visual rendering, on two 8-GPU L40S nodes (16 GPUs total). Four key designs:

1. Stage-Based Reward Design

The task is segmented into walking, placing, grasping, and turning, with four rewards:

$$r^{\text{walk}} = \exp\!\big(-4\,(\lVert p_{\text{robot}} - p_{\text{GraspObj}}\rVert - 0.45)^2\big)$$

$$r^{\text{place}} = -\lVert f_{\text{PlaceObj}}\rVert \cdot \mathbb{1}\big(\lVert p_{\text{PlaceObj}} - p_{\text{tray}}\rVert < 0.3\big)$$

$$r^{\text{grasp-z}} = \min\big(h_{\text{GraspObj}} - h_{\text{table}},\ 0.15\big), \qquad r^{\text{grasp-goal}} = \exp\!\big(-10\,\lVert p_{\text{GraspObj}} - p_{\text{goal}}\rVert^2\big)$$

$$r^{\text{turn}} = -\lvert y_{\text{robot}} - y_{\text{desired}}\rvert$$

The placing reward uses the force between the robot's fingers and the object to be placed, gated by an indicator that the object is within 0.3 m of the tray — penalizing contact force only when close to the tray, thereby encouraging stable placement.

2. Delta Action Space

Unlike much of the legged locomotion RL literature, which commonly uses absolute joint targets, VIRAL adopts a delta action space: the policy outputs increments that accumulate into the WBC command. In practice this significantly accelerates and stabilizes RL training — the ablation shows only the delta-action teacher reliably solves the task, while the absolute-action variant fails to reach high success.

3. WBC Command as an API

Rather than learning low-level motor skills from scratch, the teacher outputs high-level WBC commands. HOMIE serves as the underlying WBC controller, providing stable lower-body locomotion and diverse upper-body poses; VIRAL extends its command interface with finger actions. This "API-ification" pays off twice: it confines the action space to a safe and reliable region of humanoid motions, improving deployability, and it substantially reduces the reward engineering burden. The authors emphasize the framework is not overfit to a specific WBC policy and can be extended to other humanoid WBC controllers.

4. Reference State Initialization (RSI)

Figure 4: Frames of reference state initialization for teacher RL.

Learning long-horizon skills for high-DoF humanoids with RL typically demands heavy reward engineering and still often yields suboptimal or poor sim-to-real transfer. VIRAL instead collects 200 teleoperated simulation demonstrations and uses them as a state-initialization buffer for RL: at every episode reset, a demonstration snapshot is sampled and the scene (robot, objects, tables) is initialized accordingly, exposing the policy to diverse rewarding states long before it can reach them from scratch. Human-provided grasping and placement poses offer strong priors, so this reference-biased exploration greatly reduces reliance on brittle reward tuning. The ablation is stark: without RSI, the teacher quickly plateaus below 10% success; with RSI, it reaches nearly 95%.

Phase 2: Visual Student Distillation (64× L40S + Tiled Rendering)

Element 1: DAgger & BC Mixture

The student is distilled from the privileged teacher through a hybrid of online DAgger and behavior cloning (BC). Both share the same MSE objective, computed over a mixture of teacher- and student-induced observation distributions:

$$\rho_o \triangleq \alpha\,\rho^{\pi_{\text{teacher}}}_o + (1-\alpha)\,\rho^{\pi_{\text{student}}}_o$$

$$\mathcal{L}_{\text{distill}} = \mathbb{E}_{o_t \sim \rho_o}\Big[\ \big\lVert \pi_{\text{teacher}}(o^{\text{teacher}}_t) - \pi_{\text{student}}(o^{\text{student}}_t) \big\rVert^2_2\ \Big]$$

where $\alpha$ is the mixture ratio. The distinction between DAgger and BC lies solely in the source of observations: teacher rollouts provide clean, near-optimal demonstrations that rapidly imprint strong priors, while student rollouts expose the learner to states outside the teacher's ideal distribution — critical for error-correction robustness and for preventing compounding error at deployment. The ablation (Figure 11) shows pure BC ($\alpha=1$) yields fast loss reduction but produces a brittle policy that fails to correct its own mistakes and performs poorly in Isaac-to-MuJoCo and real-world evaluations; introducing student rollouts ($\alpha=0.5$) slows optimization slightly but substantially improves deployment success, so $\alpha=0.5$ is the default.

Element 2: Network Backbone

The student's vision backbone adopts a state-of-the-art image encoder (DINOv3) to extract high-quality RGB features, fused with proprioception available on real hardware before the policy head. The ablation shows stronger visual representations and greater capacity yield faster convergence and higher task success.

Element 3: Distributed Simulation Learning System

Large-scale visual simulation is substantially more expensive than rendering-free physics — typically at least an order of magnitude slower in throughput. VIRAL implements a customized version of TRL with Accelerate support for efficient scaling across multiple GPUs and compute nodes, preserving the simplicity of single-GPU training while enabling near-linear scaling to large clusters. Visual distillation runs on eight 8-GPU L40S nodes (64 GPUs total) with tiled rendering in Isaac Lab.

Three Things That Bridge Sim-to-Real

Figure 3: Visual randomization on image, lighting, material, and camera extrinsics for sim-to-real robustness.

1. SysID for the dexterous hand. Modern humanoids increasingly use low gear-ratio motors, reducing the need for motor-level SysID — but the Unitree G1's three-fingered dexterous hand employs high gear ratios, resulting in a substantial sim-to-real mismatch. VIRAL defines a real-world grasp-release primitive, replays the identical action sequence in simulation, and performs SysID over finger armature, stiffness, and damping to align simulated joint trajectories with real measurements.

Figure 5: System identification of the dexterous hand. Real–sim overlays (top) and joint position trajectories (bottom) before and after SysID.

2. FOV alignment and randomization. The simulator's camera intrinsics (focal length, focus distance, sensor apertures) are matched to the manufacturer's specifications. However, camera extrinsics on G1 units vary across units due to mechanical tolerances and can even drift over time on the same robot. VIRAL performs a lightweight real-to-sim extrinsics calibration by visually matching rendered and real images, and further applies extrinsics randomization during training so the student remains robust to hardware-induced viewpoint differences.

Figure 6: Real-to-sim camera extrinsics alignment. Real view versus simulated views before and after alignment.

3. Visual and simulation randomization. Image quality (brightness, contrast, hue, saturation, Gaussian noise, blur), camera extrinsics for small pose shifts, camera latency to model transmission delays, global illumination via dome-light environments, and material and color properties of floors, tables, objects, and robot components are all randomized. In the ablation (Figure 13) three dominant components are material randomization for table/floor/robot (M), dome-light randomization (D), and camera-extrinsics randomization (E): turning off all randomization causes performance to drop to 0.649 (a 35.1% decrease), and removing any single component also degrades performance — the randomizations are complementary and together form a crucial pipeline for robust transfer.

Real-World Results

Robustness: 54/59, Faster Than the Expert

The evaluation task is continuous loco-manipulation: the humanoid repeatedly walks between two tables, places an object, grasps a new object, and turns around. VIRAL succeeds in 54 out of 59 consecutive real-world trials. It is also compared with two human teleoperators, both using the same HOMIE policy for a near-apple-to-apple comparison:

The takeaway: although expert-level success remains challenging, VIRAL achieves near-expert success performance while being faster than the expert, and substantially outperforms non-experts in both reliability and efficiency — highlighting its potential to reduce human workload in assisted teleoperation settings.

Generalization: Across Every Variation Tested

Generalization is assessed by systematically varying tray start position, robot start pose, table height, lighting, table cloth, table type and color, and object category (Figure 8). Across all these variations, VIRAL consistently completes the task without additional tuning. The authors attribute this to the domain randomization used during training and the robustness of RL, which exposes the policy to diverse visual and spatial conditions.

Ablations: What Actually Matters

The two compute-scaling ablations deserve particular attention. On the teacher side (Figure 14), increasing GPUs does not merely accelerate learning — it has a pronounced effect on asymptotic performance. Early training even shows better-than-linear speedup (reaching a modest success rate of ~0.2 with 4 GPUs takes well under half the time required with 2), reflecting richer on-policy experience and more diverse rollouts. With insufficient compute (1–2 GPUs) the teacher plateaus far below the desired range and never reaches high success; 8–16 GPUs consistently drive it above 90%. On the student side (Figure 15), scaling from 1 to 64 GPUs reaches the same loss threshold dramatically sooner and raises the success curve much more steeply. Beyond speed, scaling improves training stability: more GPUs give smoother loss curves and less variance in success rate, especially in early stages when the student is most sensitive to distribution shift. The authors conclude that substantial computing is not merely a convenience but a practical requirement for reliable visual loco-manipulation distillation.

Significance and Limitations

Why it matters. VIRAL is a strong counter-example to the prevailing narrative that humanoid mobile manipulation is "just another data problem." It uses no real robot training data at all — only 200 simulated teleoperation demos for state initialization — and still reaches near-expert performance on a long-horizon walk-place-grasp-turn loop. For a domain where real data is extraordinarily expensive, this is a path with a completely different cost structure.

Three transferable engineering lessons. First, use demonstrations for state initialization rather than as imitation targets — this sidesteps the most brittle part of reward engineering in long-horizon tasks, at very low cost. Second, treat WBC as an API — confining policy output to an already-validated safe and reliable motion manifold means RL only learns "intent," not motors, cutting reward engineering while improving deployability. Third, pure BC trains a brittle policy that cannot correct itself — applicable to any team doing visual distillation: you must mix student-rolled-out states into the data distribution.

On limitations, the paper is candid in its conclusion: while sim-to-real has shown remarkable success on isolated capabilities — robust locomotion, geometric perception, rigid-body manipulation — scaling these methods to general-purpose loco-manipulation ("locomote anywhere, perceive anything, manipulate everything") exposes four critical coverage gaps that current paradigms have yet to bridge. Further boundaries are visible from the design: the task is confined to a structured loop between two tables with a limited set of object geometries and materials; the system depends on an external WBC (HOMIE) whose capability ceiling bounds VIRAL's action space; real camera extrinsics drift over time, and the lightweight calibration is one-shot, so long-horizon stability is unverified; and the 16-GPU teacher / 64-GPU student requirement means reproducing these results itself demands considerable resources.