Learning Humanoid Locomotion with Perceptive Internal Model

Paper Metadata

Title: Learning Humanoid Locomotion with Perceptive Internal Model

Authors: Junfeng Long, Junli Ren, Moji Shi, Zirui Wang, Tao Huang, Ping Luo, Jiangmiao Pang

Institution: Shanghai AI Laboratory, University of Hong Kong, Zhejiang University, Shanghai Jiao Tong University

Link: arxiv.org/abs/2411.14386

Project Page: junfeng-long.github.io/PIM

One-Sentence Summary

The Perceptive Internal Model (PIM) integrates LiDAR elevation maps into the Hybrid Internal Model's state estimation, enabling humanoid robots to train in 3 hours on an RTX 4090 and achieve zero-shot deployment on Unitree H1 and Fourier GR-1 with >90% continuous stair climbing success rate.

Background and Motivation

Unlike quadruped robots that navigate diverse terrains using "blind" policies (proprioception only), humanoid robots require accurate perception for stable locomotion due to their high degrees of freedom and inherently unstable morphology. However, incorporating perceptual signals introduces additional disturbances that may reduce robustness, generalizability, and efficiency. This is the core contradiction in humanoid locomotion control: without perception, robots fall on complex terrains; with perception, sim-to-real gaps and computational costs arise.

Existing methods mostly follow "blind" policies or multi-stage training paradigms. Blind policies only enable walking on flat surfaces and mildly uneven terrains—robots fall on stairs requiring precise footholds. Some methods rely on prior motion trajectories for human-like walking but struggle with challenging terrains. Humanoid Parkour Learning uses perception but cannot handle continuous stairs because it simulates hardware noise through an extra training phase, sacrificing perception accuracy. Multi-stage training processes are computationally expensive and complex.

Another key issue is the sim-to-real perception gap. Rendering depth images in simulation incurs computational memory and efficiency costs, and the domain gap between simulated and real depth maps is hard to bridge. Most methods directly encode depth maps or raw point clouds, which are sensitive to camera movement and noise. As the camera shakes with the robot's body during walking, depth maps become unstable and point clouds jitter, with disturbances amplified by the policy network.

The core insight: instead of using depth maps or point clouds directly, construct a robot-centered, continuously updated elevation map as the perception representation. This has three advantages: (1) the elevation map accounts for odometry and maintains a local map, letting the robot clearly perceive terrain beneath its feet without camera jitter effects; (2) simulation requires no depth rendering—directly querying terrain heights introduces minimal computational cost; (3) the elevation map is more robust to sensor movement and noise as a filtered, accumulated representation.

System Architecture

The framework builds on the Hybrid Internal Model (HIM), with PIM adding perceptual information for state prediction. Inputs include proprioceptive and perceptive information; PIM estimates state (linear velocity and latent), then a PPO policy outputs actions. Training uses ground-truth terrain heights in simulation; inference uses real LiDAR elevation maps. The elegant design: simulation directly uses ground-truth height maps (no depth rendering, no domain gap), and real elevation maps naturally align with ground-truth heights.

Figure 2: PIM framework overview. Perceptive information is integrated into the state predictor.

Observation Design

Policy observations $\mathbf{o}_t$ comprise velocity commands $\mathbf{c}_t=[v_x^c, v_y^c, \omega_{yaw}^c]$, proprioceptive information (joint position $\theta_t$, joint velocity $\dot{\theta}_t$, base angular velocity $\omega_t$, gravity direction $\mathbf{g}_t$), perceptive information $\mathbf{p}_t$ (elevation samples), and last action $\mathbf{a}_{t-1}$. Observations split into non-perceptive $\mathbf{o}_t^n = [\mathbf{c}_t, \omega_t, \mathbf{g}_t, \theta_t, \dot{\theta}_t, \mathbf{a}_{t-1}]$ and perceptive $\mathbf{o}_t^p = [\mathbf{p}_t]$. The critic accesses privileged information like linear velocity $v_t$ during training.

PIM Core Improvement

PIM concatenates current perceptive observation $\mathbf{o}_t^p$ with proprioceptive history $\mathbf{o}_{t-H:t}^n$ for next-step state estimation. Terrain plays a crucial role in robot state transitions—the same action produces vastly different state changes on flat ground vs. stairs. Without knowing the terrain, the estimator cannot accurately predict next states. PIM's symmetry regularization uses three operators: $G_o^n$ (flip proprioceptive observations), $G_o^p$ (flip perceptive observations), $G_a$ (flip actions). Policy symmetry loss:

$$\mathcal{L}_{symmetry}^{policy} = MSE(G_a(\pi(\mathbf{o}_t^n, \mathbf{o}_t^p, PIM(\mathbf{o}_{t-H:t}^n, \mathbf{o}_t^p))), \pi(G_o^n(\mathbf{o}_t^n), G_o^p(\mathbf{o}_t^p), PIM(G_o^n(\mathbf{o}_{t-H:t}^n), G_o^p(\mathbf{o}_t^p))))$$

Value symmetry loss: $\mathcal{L}_{symmetry}^{value} = MSE(V(\mathbf{o}_t^n, \mathbf{o}_t^p), V(G_o^n(\mathbf{o}_t^n), G_o^p(\mathbf{o}_t^p)))$.

The proprioceptive observation is defined as:

$$ \mathbf{o}^{n}_{t}=[\mathbf{c}_{t},\,\mathbf{\omega}_{t},\,\mathbf{g}_{t},\,\mathbf{\theta}_{t},\,\dot{\mathbf{\theta}}_{t},\,\mathbf{a}_{t-1}] $$

where the command vector $\mathbf{c}_{t}=[v_{x}^{c},v_{y}^{c},\omega_{\text{yaw}}^{c}]$, and the perception observation $\mathbf{o}^{p}_{t}=[\mathbf{p}_{t}]$ is the elevation map scan covering a $0.8m \times 1.2m$ area centered on the robot.

Elevation Map Perception Module

The elevation mapping module generates height maps with the map frame aligned to gravity. Point clouds are height-filtered to preserve ground points, then combined with odometry to produce grid height maps. Two sensor configurations tested: Mid-360 LiDAR (providing both odometry via FAST-LIO and point cloud) and Realsense T265+D435. The LiDAR setup is more robust during rapid/irregular movements. 96 points are sampled in a 0.8m×1.2m robot-centered area, with relative z-coordinates as perceptive input.

Figure 3: Terrain perception module implemented by a single LiDAR

Action Curriculum and Training

Action space curriculum: less important joints (arms, waist) start with zero range, gradually increasing. Training alternates PPO policy optimization (PIM frozen) and PIM optimization (using collected trajectories). Total training: 3 hours on RTX 4090.

System Pipeline

flowchart TB
    A[Proprioceptive
Observation History] --> D[PIM State Estimator]
    B[Elevation Map
from LiDAR/RGBD] --> C[Sample 96 Points
0.8m x 1.2m]
    C --> D
    D --> E[Estimated Velocity
+ Latent State]
    E --> F[PPO Policy Network]
    G[Current Observations] --> F
    F --> H[Joint Actions]
    H --> I[Humanoid Robot
H1 / GR-1]
    I --> A
    I --> B

Experimental Results

PIM vs HIM

PIM (perception for state estimation) outperforms HIM (no perception for estimation) in training efficiency, estimation accuracy, and traversable terrain difficulty. PIM achieves lower estimation loss faster and enables traversal of more difficult terrains, validating the core value of incorporating perception into state estimation.

Stair Climbing

The system enables humanoid robots to traverse 15cm stairs with >90% continuous success rate, surpassing existing methods that only handle simple low steps. Cross-platform validation on Unitree H1 and Fourier GR-1 demonstrates generalization without platform-specific policy design.

Figure 4: Hardware specifications of Unitree H1 and Fourier GR-1

Multi-Terrain Traversal

Beyond stairs, the system validates traversal of wooden platforms (≥0.4m), slopes (15°), and gaps. H1 can consecutively step onto platforms and jump over gaps between them.

The linear velocity tracking reward uses exponential form:

$$ r_{v}=\exp\left\{-\frac{\|\mathbf{v}_{xy}^{\text{cmd}}-\mathbf{v}_{xy}\|_{2}^{2}}{\sigma}\right\} $$

The angular velocity tracking reward:

$$ r_{\omega}=\exp\left\{-\frac{(\omega_{\text{yaw}}^{\text{cmd}}-\omega_{\text{yaw}})^{2}}{\sigma}\right\} $$

The state prediction loss is defined as:

$$ \mathcal{L}_{\text{pred}}=MSE\left(G_{a}(\pi(\mathbf{o}^{n}_{t},\mathbf{o}^{p}_{t},PIM(\mathbf{o}^{n}_{t-H:t},\mathbf{o}^{p}_{t}))),\,\mathbf{o}^{n}_{t+1}\right) $$

Reward Design

Key rewards include: feet lateral distance (penalize feet too close, weight 2.5), feet ground parallel (sample 5 points per foot, penalize height variance, weight -2.0), linear velocity tracking (weight 1.0), joint deviation penalties (arm, hip, waist), feet stumble (triggered when horizontal force exceeds 3x vertical, weight -3.0).

Limitations

Author-stated: Terrain variety, while diverse, remains limited; extreme conditions (slippery surfaces, loose gravel) untested. Elevation map quality depends on odometry precision, which may degrade in feature-poor environments.

Analysis: The 0.8m×1.2m perception range with 96 points may be insufficient for large obstacles beyond range. LiDAR adds hardware cost and weight. Ground point filtering may fail in complex occlusion scenarios. The method focuses on locomotion only; integration with manipulation policies is needed for complete embodied intelligence.

Conclusion

PIM integrates LiDAR elevation maps into HIM's state estimation for perceptive humanoid locomotion. Three key advantages: elevation map robustness to noise, minimal training cost (3 hours), and single-stage zero-shot deployment. Experiments on H1 and GR-1 demonstrate cross-platform generalization with >90% stair climbing success and multi-terrain traversal capability. PIM has potential as a foundational algorithm for future humanoid control methods.