Heavy lifting tasks via haptic teleoperation of a wheeled humanoid

Paper Metadata

Title: Whole-body Bilateral Teleoperation for Dynamic Mobile Lifting Tasks of a Wheeled Humanoid

Authors: Amartya Purushottam, Jack Yan, Christopher Xu, Joao Ramos

Institution: University of Illinois at Urbana-Champaign (UIUC)

Link: arxiv.org/abs/2505.19530

Video: youtu.be/jF270_bG1h8

One-Sentence Summary

A whole-body bilateral teleoperation framework for wheeled humanoid robots that retargets human pilot motion to the robot via an HMI, combining haptic feedback and automatic pitch compensation to lift objects up to 21% of robot mass and transport them across varying heights.

Background and Motivation

Humanoid robots can assist human workers in physically demanding environments by performing tasks requiring whole-body coordination, such as lifting and transporting heavy objects. These Dynamic Mobile Manipulation (DMM) tasks require simultaneous control of locomotion, manipulation, and posture under dynamic interaction forces. In warehouse scenarios, workers bend down to pick up heavy objects and place them on shelves—requiring simultaneous walking, grasping, and balancing.

Despite rapid advances, robots still struggle with these maneuvers. Learning-based policies show potential but need teleoperation for demonstration data collection. Motion retargeting lets the robot become an extension of the human, with the pilot using arms and body to simultaneously control manipulation and locomotion. Haptic feedback enhances the experience by letting the pilot "feel" interaction forces.

Prior work has significant gaps: bipedal platforms use trajectory optimization but only at fixed positions; Golem Krang showed locomotion and height change but no manipulation; Ballbot adjusts lean but at fixed height; CENTAURO focuses on grasp force estimation with small payloads; Reflex shows impressive loco-manipulation but is fully actuated without explicit payload disturbance handling. No prior work demonstrates teleoperated DMM with height retargeting, explicit payload handling, and immersive haptic feedback in a unified system.

System Architecture

The HMI captures the pilot's arm motion, height, pitch, and center of pressure, retargeting these to robot setpoints. The robot is modeled as a wheeled inverted pendulum with linearized dynamics:

$$\dot{\bm{q}}_{R}=\bm{Aq}_{R}+\bm{Bu}_{R}+\bm{d}_{R}$$

where $\bm{q}_R=[x_R, \theta_R, \dot{x}_R, \dot{\theta}_R]$ and $\bm{d}_R$ captures external disturbances.

Figure 1: A human operator controls the robot to lift a barbell weighing 21% of robot mass, with automatic backward lean compensation

Methods

Height Variation Locomotion Retargeting

Human height deviation $\Delta h_H$ maps to robot height change: $\frac{\Delta h_{R}}{h_{R}^{nom}}=\frac{\Delta h_{H}}{h_{H}^{nom}}$, giving desired height $h_R^{des} = h_R^{nom} + \beta_z \frac{h_R^{nom}}{h_H^{nom}} \Delta h_H$ with $\beta_z = 0.5$. The controller compensates for leg-induced disturbances via feedback linearization:

$$u=-\bm{K}_{LQR}(h_R)(\bm{q}_{xR}^{des}-\bm{q}_{xR})-\bm{B}^{\dagger}\bm{d}_w$$

where LQR gains are linearly interpolated by robot height.

Desired Pitch Estimation with Payload

Static equilibrium around the wheel axle: $F_g h_R \sin(\theta_R^*) = -F_{obj} x_{ee}$, where $x_{ee} = L_b \sin(\theta_R^*) + L_1 \sin(\theta_R^*+\phi_1) + L_2 \sin(\theta_R^*+\phi_1+\phi_2)$. Solving:

$$\theta_R^* = \tan^{-1}\left(\frac{f_1}{f_2}\right)$$

with $f_1 = -F_{obj}(L_1 \sin\phi_1 + L_2(\sin\phi_1\cos\phi_2 + \cos\phi_1\sin\phi_2))$ and $f_2 = F_g h_R + F_{obj}(L_b + L_1\cos\phi_1 + L_2(\cos\phi_1\cos\phi_2 - \sin\phi_1\sin\phi_2))$.

Figure 2: Left: Haptic force feedback creates a moment around the human ankle. Right: Robot model computes new desired pitch to counteract payload moment.

Error Dynamics Locomotion Retargeting

Robot DCM $\xi_R = \theta_R + \dot{\theta}_R/\omega_R$, human DCM $\xi_H = \theta_H + \dot{\theta}_H/\omega_H$, with $\omega_H = \sqrt{g/h_H}$, $\omega_R = \sqrt{g/h_R}$. Desired DCM with payload: $\xi_R^* = \theta_R^* + \dot{\theta}_R^*/\omega_R$. Pilot controls error: $\frac{\dot{\xi}_R - \dot{\xi}_R^*}{\omega_R} = \frac{\dot{\xi}_H}{\omega_H}$.

Haptic Feedback

$$F_{xH}^{HMI}=\gamma_{H}(\xi_{R}-\xi_{H})+\frac{\gamma_{H}}{\gamma_{R}}F_{xR}^{ext}$$

First term: DCM error (balance cue). Second term: scaled external force (interaction cue).

System Pipeline

flowchart TB
    A[Human Pilot
Pitch + Height + Arms] --> B[HMI Motion Retargeting]
    B --> C[Desired Robot Height
+ Pitch + Arm Targets]
    C --> D{Compensation Mode}
    D -->|Auto| E[Estimate Desired Pitch
from Payload Mass]
    D -->|Manual| F[Haptic Feedback:
Full Moment to Pilot]
    E --> G[Gain-scheduled LQR
+ Feedback Linearization]
    F --> G
    G --> H[Wheeled Humanoid Robot]
    H --> I[End-effector Wrench
+ DCM Error]
    I --> J[Haptic Feedback to Pilot]
    J --> A

Experimental Results

Three Control Strategies

Three strategies compared: (1) velocity mapping with auto compensation; (2) DCM pitch mapping with auto compensation; (3) fully manual DCM mapping with full moment feedback $F_{fb} = \frac{\gamma_H}{\gamma_R} M_{ext}^y$. Tasks: lifting and transporting 2.5kg barbells and boxes (21% of robot mass) between elevated and lowered positions. Automatic lean compensation improved performance and reduced pilot effort. DCM mapping outperformed velocity mapping in DCM tracking but velocity mapping was better for precise positioning.

Figure 4: Left: Pilot transports 2.5kg box between heights. Right: DCM pitch mapping (blue) vs velocity mapping (red) DCM tracking.

Robot state-space model

$$ \bm{\dot{q}}_{R}=\bm{Aq}_{R}+\bm{Bu}_{R}+\bm{d}_{R} $$

Key Hyperparameters

Parameter	Value	Description
Learning rate	1e-4	Adam optimizer
Batch size	256	Training batch
Discount $\gamma$	0.99	Reward discount
Epochs	5000	Training iterations

Limitations

Author-stated: Monocular FPV cameras lack depth perception, causing gripper-object alignment errors. Some object geometries require precise alignment.

Analysis: Assumes known object mass (within 10%); real scenarios need online inertial parameter identification. Wheeled platform limited to flat ground. Limited haptic channels may exceed human perceptual bandwidth for complex multi-axis interactions. High cognitive load requires extensive pilot training.

Conclusion

This teleoperation framework enables dynamic mobile manipulation on a wheeled humanoid, integrating height retargeting, explicit payload handling, and haptic feedback. The pilot controls whole-body posture via motion retargeting while receiving force cues reflecting interaction forces and balance demands. Heavy lifting experiments show automatic lean compensation improves performance and reduces pilot effort. Velocity and pitch control provide complementary benefits across task phases. This work provides an important data collection foundation for teleoperation-driven robot learning.