HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL

Turning a generalist VLA into a full-body humanoid pilot

One-line summary: HAF turns an off-the-shelf flow-matching VLA into a humanoid whole-body loco-manipulation policy by splitting action denoising into a locomotion→waist→manipulation hierarchy (HAF-VLA), then fine-tunes it online through a frozen-backbone RL loop that only edits an 8-coefficient DCT noise subspace (HAF-Steer). On seven real tasks with the TienKung humanoid, average normalized score rises from 53.3% (π_0.5) to 70.5%.

Figure 1: The HAF framework and the seven loco-manipulation tasks used for evaluation.

Paper: HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL
Authors: Langzhe Gu, Chengkai Hou, Meng Li et al. (Peking University · Beijing Innovation Center of Humanoid Robotics)
Links: arXiv 2608.16837 · Project page · Code not released at time of writing

Background: why generalist VLAs stumble on humanoids

Generalist vision-language-action (VLA) models such as π₀, π_0.5 and GR00T produce an action chunk by denoising from Gaussian noise in a single pass, with every body part solved at once. On a fixed-base arm this works well. On a humanoid it collides with physics: locomotion, torso posture and bimanual manipulation are not interchangeable dimensions — an unstable base produces erratic compensatory arm motion that destroys manipulation accuracy. The paper frames this as the core obstacle: single-stage generation "entangles all body movements" and ignores the kinematic dependency structure.

Two prior routes exist, and both are expensive. One is humanoid-specific pretraining on large embodiment datasets, which must be re-run for every new platform. The other is online RL on the raw VLA, which is computationally heavy and unsafe on a 1.7-meter robot. Existing latent-noise RL fixes the safety problem but either optimizes the full high-dimensional temporal noise or repeats one noise vector across the horizon — trading efficiency for temporal expressiveness. HAF attacks both problems with two complementary components.

HAF-VLA: three stages, one shared expert

The action chunk is a sequence $A_t=[a_t,\ldots,a_{t+H-1}]\in\mathbb{R}^{H\times D}$ with $H=100$ steps (the robot executes the first 40 before re-querying). Each per-step action decomposes into four kinematic subsets: $a_t=[a_t^{\mathrm{move}},a_t^{\mathrm{head}},a_t^{\mathrm{waist}},a_t^{\mathrm{manip}}]$. From these the authors build nested cumulative index sets:

$$\mathcal{A}_t^{1}=\mathcal{A}_t^{\mathrm{move}}\cup\mathcal{A}_t^{\mathrm{head}},\quad \mathcal{A}_t^{2}=\mathcal{A}_t^{1}\cup\mathcal{A}_t^{\mathrm{waist}},\quad \mathcal{A}_t^{3}=\mathcal{A}_t^{2}\cup\mathcal{A}_t^{\mathrm{manip}}$$

Stage 1 denoises locomotion and head gaze; Stage 2 adds waist posture; Stage 3 activates the full space. Crucially the sets are nested rather than disjoint, so later stages may refine earlier dimensions instead of freezing them. All three stages reuse one action expert $F_\theta$, one visual-language prefix cache $P_t=F_{\mathrm{prefix}}(o_t,\ell)$, and independent Gaussian noise per stage, distinguished by a learned stage embedding $e_s$:

$$A_t^1=F_\theta(\epsilon_t^1\mid P_t,e_1),\quad A_t^2=F_\theta(\epsilon_t^2\mid P_t,C_t^1,e_2),\quad A_t^3=F_\theta(\epsilon_t^3\mid P_t,C_t^1,C_t^2,e_3)$$

The operators $C_t^1, C_t^2=\mathrm{Cache}_\theta(\cdot)$ re-encode each stage's denoised output into a KV cache consumed by later stages — this is how "stand up straight first, then reach" propagates through the network. Only $A_t^3$ is executed.

Training uses stage-wise flow matching. With a binary mask $m_s$ for stage $s$, noisy input and target are masked into the global action space:

$$X_{\tau^s,t}^{s}=\left[(1-\tau^s)\epsilon_t^s+\tau^s A_t\right]\odot m_s,\qquad u_t^{s}=(A_t-\epsilon_t^s)\odot m_s$$

$$\mathcal{L}_{\mathrm{HAF}}=\mathbb{E}\left[\sum_{s=1}^{3}\frac{1}{HD}\left\|v_\theta(X_{\tau^s,t}^{s},\tau^s;h_t^s)-u_t^s\right\|_F^2\right]$$

Inactive dimensions get zero target velocity instead of being dropped, and teacher forcing builds the cross-stage caches from ground truth during training. Deployment runs 10 denoising steps per stage — about 0.12 s for all three stages on one RTX 5090 — with receding-horizon execution of the first 40 steps.

flowchart TD
    O["Observation o_t + language l"] --> P["Prefix cache P_t
(computed once)"]
    P --> S1["Stage 1: locomotion + head
A1 = F(e1 | P, e_1)"]
    S1 --> C1["Cache C1"]
    C1 --> S2["Stage 2: + waist
A2 = F(e2 | P, C1, e_2)"]
    S2 --> C2["Cache C2"]
    C2 --> S3["Stage 3: + manipulation
A3 = F(e3 | P, C1, C2, e_3)"]
    S3 --> R["Execute first 40 of 100 steps"]

The HAF-VLA generation pipeline: one shared expert, three nested action masks, cross-stage KV caches.

Figure 2: HAF-VLA progressively expands the active action space across three stages.

HAF-Steer: RL in an 8-coefficient spectral noise space

Flow matching is deterministic and invertible: $A_t=F_\theta(\epsilon_t\mid\zeta_t)$. Instead of retraining the backbone, HAF-Steer treats the initial noise $\epsilon_t$ as the control interface. A demonstrated action is inverted by numerically integrating the frozen flow backwards, then compressed with a truncated DCT:

$$\epsilon_i^*=F_\theta^{-1}(A_i^*\mid\zeta_i^*),\quad c_i^*=\mathrm{DCT}_K(\epsilon_i^*),\quad z_i^*=\frac{c_i^*-\mu_c}{\sigma_c+\delta}$$

With $K=8$ kept coefficients (out of $H=100$ steps), exploration shrinks from $H\times D$ to $K\times D$, and the low-frequency basis functions act as a physical smoothness prior. The authors report that the repeated-single-noise trick used by prior steering work (DSRL) caused body jerking and unsafe transients on the real robot — keeping several low-frequency modes avoids that. A stochastic actor $\pi_\psi$ is initialized by behavior cloning in this space:

$$\mathcal{L}_{\mathrm{BC}}=\mathbb{E}_{(x_i,z_i^*)\sim\mathcal{D}_{\mathrm{off}}}\left[\left\|\mu_\psi(x_i)-z_i^*\right\|_2^2\right]$$

then refined with mixed offline-online SAC using a sparse terminal reward ($r=1$ only on successful episode ends). The policy loss blends the SAC objective over mixed batches with a BC regularizer applied to offline data only:

$$\mathcal{L}_\pi=\mathbb{E}_{x_i\sim\mathcal{B},z_i\sim\pi_\psi}\left[\alpha\log\pi_\psi(z_i\mid x_i)-\min_j Q_{\omega_j}(x_i,z_i)\right]+\lambda_{\mathrm{BC}}\mathbb{E}_{(x_i,z_i^*)\sim\mathcal{B}_{\mathrm{off}}}\left[\left\|\mu_\psi(x_i)-z_i^*\right\|_2^2\right]$$

The offline sampling ratio decays during training to hand control over to live robot experience. Throughout, the entire VLA backbone stays frozen — only the spectral actor, critics and entropy temperature update. At test time: $z_t\sim\pi_\psi(\cdot\mid x_t)$, $c_t=\mu_c+(\sigma_c+\delta)\odot z_t$, $\epsilon_t=\mathrm{IDCT}_K(c_t)$, $A_t=F_\theta(\epsilon_t\mid\zeta_t)$. On HAF-VLA, steering is applied to Stage 3's initial noise only.

Figure 3: HAF-Steer. Flow reversal maps demos back to noise, DCT compresses them, and SAC learns in the compact spectral subspace while the VLA stays frozen.

flowchart LR
    D["Offline demos A*"] --> INV["Flow reversal
F_theta^-1 (frozen)"]
    INV --> DCT["DCT_K, K=8
+ normalize"]
    DCT --> BC["BC init of spectral actor"]
    BC --> SAC["Mixed offline-online SAC
sparse terminal reward"]
    SAC --> DEC["IDCT decode -> eps_t"]
    DEC --> GEN["Frozen VLA generator -> A_t"]
    GEN --> ROBOT["Real humanoid rollout"]
    ROBOT -->|"new transitions"| SAC

The offline-to-online loop: only the spectral actor and critics are trained.

Experiments: seven household tasks on TienKung 2.0/3.0

Data comes from isomorphic teleoperation — master arms for bimanual work, a joystick for locomotion/waist, an IMU for head gaze — 120 trajectories per task. Baselines are ACT, π_0.5, GR00T N1.7 and Cosmos Policy. The metric is a normalized milestone score, averaged over 10 rollouts.

Task	HAF-VLA	π0.5	GR00T N1.7	Cosmos	ACT
Laundry Loading	66.7	53.3	40.0	0.0	10.0
Clothes Retrieval	53.3	53.3	33.3	26.7	23.3
Table Tidy	80.0	70.0	40.0	16.7	23.3
Basket Transfer	63.3	50.0	43.3	33.3	16.7
Toy Storage	80.0	53.3	30.0	40.0	23.3
Ball Tossing	56.7	33.3	36.7	3.3	30.0
Box Transfer	93.3	60.0	43.3	73.3	50.0
Average	70.5	53.3	38.1	27.6	25.2

Table 1: Normalized task scores across the seven loco-manipulation tasks.

HAF-VLA wins or ties on all seven tasks and lifts the average by 17.2 points. Gains concentrate where locomotion feeds into manipulation — Box Transfer (+33.3), Ball Tossing (+23.4), Laundry Loading (+13.4). The authors note π_0.5 and GR00T drift while walking, and Cosmos Policy's higher inference latency hurts closed-loop performance.

Ablation on Laundry Loading isolates the hierarchy itself:

Variant	Stage design	Steps	Score (%)
HAF-VLA	locomotion/head → +waist → +manip	30	66.7
All-Joint Denoising	all joints at every stage	30	53.3
Arm-First Hierarchy	manip → +waist → +locomotion/head	30	50.0
π0.5	single stage	30	20.0

Table 2: Ablation on Laundry Loading. Order matters as much as structure.

Two findings stand out. First, reversing the order (arm-first) drops to 50.0 — worse than the full hierarchy and barely above all-joint denoising, confirming that base and posture must be settled before fine manipulation. Second, plain π_0.5 at 30 steps scores only 20.0: more denoising steps increase latency (0.075 s → 0.115 s), which amplifies the temporal mismatch of already-drifting locomotion predictions. HAF-VLA tolerates the same latency budget far better.

Robustness tests add an unseen chair on the Laundry Loading path and a 20 cm backward start shift for Clothes Retrieval; HAF-VLA scores 40.0 vs 26.7 and 43.3 vs 36.7 against π_0.5. For HAF-Steer, on Toy Storage and Basket Transfer with in-distribution and 30 cm-shifted OOD goal tables, HAF-Steer improves every ID/OOD cell for both π_0.5 and HAF-VLA backbones, with best-or-tied-best results in three of four settings; DSRL's repeated-noise variant produced unsafe motions and was terminated early.

Figure 5: The seven real-world tasks demand walking, bending, squatting, carrying and throwing.

Limitations

The authors themselves flag two. The three-stage pipeline triples denoising computation and adds deployment latency — acceptable on an RTX 5090 today, but a real constraint for faster control loops or smaller hardware. Second, the latent RL module inherits the base VLA's priors: when the frozen generator's noise space simply cannot express a correction (extreme unseen scenarios), HAF-Steer cannot recover. From a reviewer's angle, the OOD tests also stay close to the demo distribution (a 20–30 cm shift), and the evaluation fleet is small — 10 rollouts per condition on a single platform family.

Conclusion

HAF's bet is that humanoid adaptation is mostly a structural problem, not a scale problem: respect the kinematic dependency order during generation, and respect the backbone's smoothness prior during RL. The result is a cheap transfer recipe — no humanoid pretraining, no backbone gradient — that moves a generalist VLA from "can operate an arm" to "can walk, bend and manipulate" on a real humanoid. If the flow-noise interface holds up at scale, steering frozen foundation policies through small spectral knobs may become a standard deployment tool.

Golden line: "Stand up first, reach later — and never touch the backbone, just tune the noise."