Architecture Is All You Need: Diversity-Enabled Sweet Spots for Robust Humanoid Locomotion

Architecture Is All You Need: Diversity-Enabled Sweet Spots for Robust Humanoid Locomotion

arXiv:2510.14947 (2025-08) | Domain: Humanoid locomotion / layered control | Platform: Unitree G1 | IsaacSim 4096 parallel envs | Single RTX 4090

One-Sentence Summary

The paper argues and empirically validates that robust humanoid locomotion requires a layered multi-rate control architecture (LCA)—a fast proprioceptive stabilizer coupled with a compact low-rate perceptual policy—trained via a two-stage curriculum (blind stabilizer pretraining then perceptual fine-tuning); on a Unitree G1, two-stage layered policies succeed on stair and ledge tasks where one-stage perceptual policies fail, demonstrating that architectural separation of timescales, not network scale or complexity, is the key enabler of robustness.

Background and Motivation

Robust humanoid locomotion over unstructured terrain is as old as the platform yet unsolved. Terrain sensing is partial and noisy, contact events are discontinuous, and controllers must react faster than perception can resolve detail. Decades of guidance-navigation-control (GNC) practice suggest a simple lesson: robustness emerges when fast low-level stabilization is paired with slower long-horizon navigation. The canonical example is aerospace GNC: a slow semantic guidance layer chooses where to go; an intermediate-rate trajectory-generation layer turns goals into feasible references; a fast feedback control layer tracks references and rejects disturbances. This "slow and flexible" above "fast and rigid" pattern with well-defined interfaces recurs across robotics and biological sensorimotor systems.

The paper takes this to its logical extreme, arguing that for high-dimensional perception-conditioned control, layered control architecture (LCA) is the primary driver of robustness. Sophisticated models, learned world representations, or intricate reward shaping help extract maximum absolute performance but are not necessary for task success when the stack itself is well-posed. In a well-posed LCA, information flows through narrow interfaces: references descend (planner→controller), tracking error or status ascends (controller→planner). Crucially, layers operate at different time scales—both reducing computational burden and improving robustness by letting each layer specialize where it is most effective.

The separation of layers in an LCA, together with heterogeneous objectives and information, enables "diversity-enabled sweet spots" (DeSS): the combined stack can outperform any single monolithic component tuned in isolation. For perception-conditioned humanoid walking, LCA implies a minimal yet sufficient stack: (i) a compact local-perception navigation encoder updating at moderate rate to construct a latent space reflecting long-horizon terrain geometry; (ii) a fast stabilizer using proprioception to condition on this geometry and contend with contact variability.

Figure 1: A humanoid robot trained to traverse complex terrain by combining perception and fast proprioception. Using these inputs effectively requires structured architecture for performant robust results.

Method

Optimization Analysis: One-Stage vs Two-Stage

The one-stage RL formulation of global optimal control:

$$\theta^* = \max_\theta \mathbb{E}\left[\sum_k \gamma^k r(s_k, a_k \mid \theta)\right]$$

attempts to solve the global optimum but is highly sensitive to initial conditions. The two-stage approach splits parameters into $\theta = [\theta_x, \theta_y]^T$ (fast $\theta_x$, slow $\theta_y$), solving the fast layer first:

$$\theta_x^\dagger = \max_{\theta_x} \mathbb{E}\left[\sum_k \gamma^k r(s_k, a_k \mid \theta_x, \theta_{y,0})\right]$$

Stage 1 removes perception (no $\theta_{y,0}$ dependence), allowing a simpler optimization more likely to find a satisfactory local maximum. Stage 2 jointly optimizes:

$$\theta_x^*, \theta_y^* = \max_{\theta_x, \theta_y} \mathbb{E}\left[\sum_k \gamma^k r(s_k, a_k \mid \theta_x, \theta_y)\right] \quad \text{s.t.} \quad \theta_{x,0} = \theta_x^\dagger$$

Under strictly concave reward landscapes the two-stage matches the one-stage, but in highly non-concave landscapes, the good Stage-1 initialization makes the optimization less susceptible to bad local maxima. Two-stage training makes the network structurally assume layered control roles, even if the network architecture is identical to the one-stage.

Figure 2: Training and deployment overview—both actor and critic are two-stage architectures with their own perception encoders; the actor receives noisy heightmaps, the critic perfect information; at deployment a depth image is filtered through the trained encoder and the actor combines it with proprioception history to determine actions.

Observations and Normalization

Let $q$ be joint positions, $\dot{q}$ joint velocities, $g_b$ projected gravity in body frame, $\omega_b$ base angular velocity, $a_{k-1}$ the last action, $u$ commanded planar velocity. The proprioceptive observation is a $K$-length history $o_k = [q_k, \ldots, q_{k-K}, \dot{q}_k, \ldots, \omega_{b,k}, \ldots, g_{b,k}, \ldots, u, a_{k-1}]$. The actor observation is $o_k^\pi = [o_k, H^\pi]$, where $H^\pi \in \mathbb{R}^{11 \times 11}$ is a noisy sparse heightmap covering 1.0m×1.0m around the robot. The critic observation $o_k^V = [o_k, v_{\text{base}}, H^V]$ includes world-frame body velocities and a larger, more accurate 1.5m×1.5m zero-noise heightmap. Both heightmaps are normalized by subtracting the grid mean and clipping to $[-1, 1]$; zero-centering removes steady-state sim-to-real offsets.

Network Architecture

Two main components: a perception encoder (encoding the heightmap into a latent) and the primary actor network (combining latent perception with proprioception to determine actions). The encoder is a small CNN or MLP mapping $H \in \mathbb{R}^{N \times N}$ to $z_H \in \mathbb{R}^{d_H}$. The actor is an LSTM or MLP (hidden layers {512, 256, 128}) outputting joint-position setpoints tracked by PD controllers. Ablating CNN vs MLP tests whether spatial encoding outperforms a simpler model even at the 11x11 heightmap scale.

Reward Design

Rewards include phase-contact consistency, foot-strike cost, foot-sliding cost, foot-orientation (flatness) cost, foot-clearance cost, and standard locomotion rewards. The phase-contact consistency reward uses XNOR agreement:

$$r_{\text{phase}}(k) = 2 - \sum_{i \in \{L,R\}} |c_i(k) - s_i(k)|$$

The foot-strike cost penalizes lateral ground-reaction forces: $r_{\text{strike}} = \sum_i \|\mathbf{F}_i^{xy}\|_2$. The foot-sliding cost suppresses planar slip during stance: $r_{\text{slide}} = \sum_i \mathcal{C}_i(k) \|\mathbf{v}_i^{xy}(k)\|_2^2$. The foot-orientation cost encourages flat feet in contact: $r_{\text{orient}} = 1 - \exp(-k_\theta \sum_i \mathcal{C}_i(k) |\theta_{i,\text{pitch}}|)$, $k_\theta=25$. The total reward is a weighted sum:

$$r = r_{\text{locomotion}} + 0.5\, r_{\text{phase}} - r_{\text{strike}} - 0.2\, r_{\text{slide}} - r_{\text{orient}} + r_{\text{clear}}$$

Two-Stage Curriculum

Stage 1 (blind stabilization): set $H \equiv 0$ for the actor (critic still has full information), train in an environment of quarter each up-stairs/down-stairs/uneven/flat, emphasizing stabilization. Stage 2 (perception-critical): re-enable $H$, allowing the robot to make longer-horizon plans based on local terrain—conditioning the blind policy on perceived surroundings.

Figure 3: Layered vs monolithic architectures—the network architecture may be identical, but two-phase training lets them assume the layered control structure.

Experimental Results

Simulation

Trained in IsaacSim on a single RTX 4090 with 4096 parallel environments, asymmetric actor-critic to 40000 steps. Seven variants compared: a blind baseline, three one-stage perception models, three two-stage models. Training batches are balanced 25% each up-stairs/down-stairs/uneven/flat.

On the medium (in-distribution) setting all policies achieve near-parity. But OOD effects are more revealing: Stairs OOD—two-stage variants have ~3x lower contacts/step than one-stage, closer to the blind baseline; the mechanism is that under noisy exteroception two-stage policies fall back to the robust blind stabilizer learned in Stage 1, while one-stage rely more on the heightfield for both planning and stabilization, leading to occasional poor foot placements. Uneven terrain OOD—two-stage outperforms one-stage by ~10 percentage points in success, because unstructured terrain is hard to memorize and robustness requires both fast stabilization and longer-horizon placement—capabilities explicitly separated and co-trained in two-stage but entangled in one-stage.

Setting	One-Stage vs Two-Stage	Key Difference
Medium (in-dist)	Near parity	Residuals within run-to-run variance
Stairs OOD	Two-stage 3x lower contacts/step	Two-stage falls back to blind stabilizer
Uneven terrain OOD	Two-stage ~10pp higher success	Separates stabilization from long-horizon placement

Table 1: Simulation OOD results summary—the benefits of architectural separation manifest out-of-distribution.

Hardware

On a Unitree G1, four tasks: stair ascent, stair descent, hinged ledge, soft ledge. Stairs are three steps (riser ~18cm) with ~25° horizontal skew, primarily stressing navigation (longer-horizon footstep/velocity planning). Ledge tasks have 36cm elevation change with transient compliance (hinged: plank on a pivot; soft: gym mat), primarily stressing control (fast stabilization under transients). Five trials per task per policy.

Key observations: (1) the one-stage MLP underperforms across terrains—footstep selection often looks reasonable but stabilization degrades quickly, consistent with over-reliance on noisy heightfields for low-level control; (2) the blind policy transfers reasonably well and is strong on ledge (control-dominant) tasks but fails more on stairs requiring precise edge-aware placement; (3) two-stage policies (MLP and CNN+MLP) perform similarly and robustly on both navigation- and control-dominant tasks—supporting the central claim: once the layered structure is in place, the specific encoder and backbone choice is of lesser importance.

Figure 4: Training rewards across policies—all perform largely identically during training; differences manifest only at OOD deployment.

Figure 5: Hardware experiment scenarios and results.

Policy Variant	Perception	Training	Stairs OOD	Uneven Terrain OOD
Blind baseline	none	no perception throughout	fails on stairs (no edge awareness)	relies on stabilization
One-stage MLP	heightmap	perception throughout	high contacts/step	~10pp lower success
One-stage CNN	heightmap	perception throughout	similar to one-stage MLP	similar
Two-stage MLP	heightmap	blind→perception	3x lower contacts/step	~10pp higher success
Two-stage CNN+MLP	heightmap	blind→perception	3x lower contacts/step	~10pp higher success

Table 2: Five policy variants under OOD settings—two-stage significantly outperforms one-stage regardless of encoder choice.

From the deeper logic of DeSS theory, the "diversity-enabled sweet spot" core is that combining heterogeneous components can outperform any monolith. In LCA, the fast layer (stabilizer) and slow layer (navigation) have different information budgets and rates: the fast layer processes high-rate proprioception with a narrow view (only current state history), the slow layer processes low-rate perception with a longer horizon (heightmap encoding future terrain). This heterogeneity lets the combined stack cover both "fast reaction" and "long-horizon planning," while a monolithic network operating at a unified rate and budget struggles to handle both. Two-stage training explicitly shapes this heterogeneous division of labor by first solidifying the fast layer then introducing the slow, avoiding the entangled degradation of both capabilities in one-stage.

From the non-linear-landscape perspective of two-stage optimization, the paper's core argument can be further elaborated. The humanoid-locomotion reward landscape is highly non-concave, full of bad local maxima (e.g., the mediocre "stand without stepping" solution, the overfit "memorize terrain via perception" solution). Stage-1 blind training simplifies the optimization by removing perception, letting the fast layer first find a robust stabilization solution (a good local maximum); when Stage 2 introduces perception with this initialization, the optimization tends to do perception augmentation near the stabilization solution rather than re-searching. This explains why two-stage can "fall back" to the blind stabilizer under OOD—because Stage-1's stabilization solution is the optimization's main attractor, and Stage 2 merely overlays perception conditioning on it.

The hardware task design cleverly separates navigation-dominant from control-dominant capabilities. Stair tasks (narrow steps with horizontal skew) primarily test long-horizon footstep planning—one must perceive stair edges in advance and plan foot placement; ledge tasks (hinged plank and soft mat) primarily test fast stabilization under transient disturbances—perception sees a nominal ledge but actual contact compliance differs, and the stabilizer must react fast. The blind policy is strong on ledges (control-dominant) but weak on stairs (lacks navigation); one-stage is weak on both (stabilization degrades); two-stage is strong on both—this crossover pattern precisely validates the claim that layered architecture simultaneously covers navigation and control.

Compared to the end-to-end trend, this work takes a clear and controversial stance. End-to-end learning advocates letting networks learn integrally from raw perception to joint torques, theoretically discovering strategies humans never designed. But the paper's empirical evidence shows that on the high-dimensional non-concave problem of unstructured-terrain locomotion, monolithic end-to-end networks easily fall into bad local maxima where stabilization and navigation entangle and degrade, while layered architecture uses structural priors to guide optimization toward better solutions. This does not deny end-to-end's value but points out that in safety-critical real-time control, structured layered design may be more reliable than pure end-to-end—especially when deployment environments exceed the training distribution.

System Architecture Diagram

flowchart TB
  subgraph Slow["Slow Layer: Navigation (low rate)"]
    HM["Heightmap H
11x11 noisy 1.0m"]
    ENC["Perception encoder
CNN/MLP -> z_H"]
    NAV["Navigation latent
long-horizon terrain geometry"]
  end
  subgraph Fast["Fast Layer: Stabilizer (high rate)"]
    PRO["Proprioception history o_k
q/dot_q/omega/g_b/u/a"]
    ACT["Actor network
LSTM/MLP {512,256,128}"]
    PD["PD controller
joint position setpoints"]
  end
  subgraph Train["Two-Stage Curriculum"]
    S1["Stage 1 blind stabilization
H=0 train fast layer only"]
    S2["Stage 2 perception-critical
enable H joint training"]
  end
  HM --> ENC --> NAV --> ACT
  PRO --> ACT --> PD
  S1 --> ACT
  S2 --> ENC
  S2 --> ACT

Limitations

Guidance layer assumed given. The minimal LCA implements only the two-layer core (stabilizer + navigation); the guidance layer (higher-level goal decision-making) is assumed given. A complete GNC stack still needs a top-level semantic guidance layer, not addressed here.

Minimal perception interface. Only an 11x11 local heightmap is used as the perception interface; more complex perception (e.g., RGB semantics, long-range terrain prediction) is not handled. While this proves the sufficiency of a minimal interface, more complex scenes may need richer perception.

Perception latency not modeled. For simplicity and consistency, perception delays or latency modeling are not included, but real-deployment perception latency may affect the effectiveness of the multi-rate timescale separation.

Limited theoretical guarantees for two-stage optimization. While intuitively the two-stage provides a good initialization avoiding bad local maxima, the paper gives no rigorous convergence guarantee; the advantage in non-concave landscapes is primarily empirical.

Summary and Outlook

The paper argues and validates that the key to robust humanoid locomotion is not model complexity but architectural layering—a fast proprioceptive stabilizer and a slow compact perceptual policy trained via a two-stage curriculum succeed on Unitree G1 tasks where one-stage fails. The core insight is "diversity-enabled sweet spots": different layers process information at different rates and budgets, and the combined stack outperforms any monolithic component. Once the layered structure is in place, the specific encoder and backbone choice matters less—explaining why simple architectures with layered training match recent methods relying on richer models.

From a broader perspective, this work is a powerful counterpoint to the current "end-to-end monolith" trend. It reminds the robotics community that the layered wisdom of biology and classical GNC accumulated over decades should not be obscured by the convenience of end-to-end learning. The robustness of layered architecture comes from information bottlenecks and timescale separation—the fast layer is not contaminated by slow noise, the slow layer is not disturbed by fast transients. This "structure before scale" idea may be more practically valuable than stacking larger models. As humanoid robots move into real unstructured environments, the layered multi-rate stack may be a more reliable deployment paradigm than monolithic networks.

Golden Lines

"Architectural separation of timescales, rather than network scale or complexity, is the key enabler of robust perception-conditioned locomotion."

"Robustness comes from information bottlenecks and timescale separation—the fast layer is not contaminated by slow noise, the slow layer is not disturbed by fast transients."