PAPER DEEP DIVE
A Replay-Constrained Simulation Framework for Personalization of Powered Knee--Ankle Prosthesis Controllers
Personalization of impedance controllers for powered prosthetic legs is critical to accommodating individual gait biomechanics but remains challenging. Existing methods rely on time-intensive human-in-the-loop exploration and/or constrain optimization to low-dimensional, single-joint parameter subspaces. Sim-to-real transfer has enabled high-dimensional locomotion control for legged robots, but in assistive device control the human partner remains un-modelable. We present a replay-constrained simulation framework: a MuJoCo-based simulator reproduces prosthetic knee-ankle dynamics while replaying recorded hip kinematics and feedback-based ground reaction forces from individual walking data, bypassing the need to model complex human neuromuscular control mechanisms. We demonstrate the framework with a deep reinforcement learning policy that personalizes phase-dependent stiffness, damping, and equilibrium angle at both joints simultaneously, maximizing a biomimicry-based reward computed solely from onboard prosthesis measurements. Experiments with three participants with transfemoral amputation during level-ground walking at 0.8~m/s demonstrate strong simulation-to-hardware predictive validity (Pearson $r=0.96$--$0.997$). The best-performing policy on hardware was consistently predicted within the top five simulation policies for all participants. The learned controllers improved overall biomimicry rewards by 42--59\% relative to the unpersonalized baseline. The framework supports scalable high-dimensional personalization of powered prosthetic legs and is amenable to extension to higher-dimensional controller parameterizations such as neural-network controllers.
One-Sentence Summary
A replay-constrained simulation framework where MuJoCo replays recorded hip kinematics and feedback-based ground reaction forces, bypassing human neuromuscular modeling, enabling DRL to personalize phase-dependent stiffness, damping, and equilibrium angle at both knee and ankle simultaneously, improving biomimicry rewards by 42-59% with sim-to-hardware Pearson r=0.96-0.997.
Figure 1 — Replay-constrained simulation: replay individual hip kinematics and ground reaction forces, simulating prosthesis knee-ankle dynamics.
1. Background and Motivation
Approximately 2.3 million individuals with limb loss in the US, projected to double by 2060. Powered knee-ankle prostheses can improve gait biomechanics, reduce metabolic demand, and achieve net-positive joint work, but realizing these benefits depends on personalized control strategies adapted to user-specific anatomy, socket fit, alignment, and compensatory patterns.
Existing limitations: manual tuning takes ~20 minutes exposing users to suboptimal controllers; HIL-RL automates personalization but is limited to low-dimensional discrete FSM parameters (12 for knee-only); CMA-ES/Bayesian optimization face hardware iteration bottlenecks. No method enables full-dimensional, multi-joint, continuous phase-dependent impedance personalization.
2. Core Method
2.1 Replay-Constrained Simulation
Core innovation: MuJoCo simulator replays recorded hip kinematics and feedback-based ground reaction forces from individual walking data, simulating prosthesis knee-ankle dynamics, bypassing the need to model complex human neuromuscular control mechanisms.
Impedance control equation in the simulator:
$$\tau_j(\phi) = K_j(\phi) \left[\theta_{eq,j}(\phi) - \theta_j\right] + D_j(\phi) \dot{\theta}_j$$
where $\phi$ is the gait phase variable, $K_j(\phi)$, $D_j(\phi)$, $\theta_{eq,j}(\phi)$ are phase-dependent stiffness, damping, and equilibrium angle for joint $j$.
2.2 Deep RL Personalization
DRL policy simultaneously optimizes phase-dependent stiffness, damping, and equilibrium angle at both knee and ankle (high-dimensional parameter space), maximizing a biomimicry-based reward computed solely from onboard prosthesis sensor measurements.
Figure 2 — Train multiple policy parameters in simulation, select best for hardware deployment.
flowchart TB
A["Recorded Walking Data
Hip kinematics + GRF"] --> B["Replay-Constrained MuJoCo Sim"]
B --> C["Prosthesis Dynamics
Knee-Ankle"]
C --> D["DRL Policy
φ-dependent K, D, θ_eq"]
D --> E["Biomimicry Reward
(onboard sensors only)"]
E -->|train| D
D --> F["Top-5 Sim Policies"]
F --> G["Hardware Deployment
with transfemoral amputees"]
G --> H["Best Hardware Policy"]
style B fill:#e0e7ff,stroke:#2563eb
style D fill:#fef3c7,stroke:#d97706
style H fill:#dcfce7,stroke:#16a34a
3. Experimental Results
3 participants with transfemoral amputation, level-ground walking at 0.8 m/s.
| Metric | Participant 1 | Participant 2 | Participant 3 |
|---|---|---|---|
| Sim-HW Pearson r | 0.997 | 0.96 | 0.98 |
| Top-5 contains best | ✓ | ✓ | ✓ |
| Biomimicry improvement | +59% | +42% | +51% |
Best hardware policy consistently predicted within simulation Top-5, validating sim-to-hardware predictive validity.
4. Main Contributions
- Replay-constrained simulation: Replays hip kinematics and ground reaction forces, bypassing human neuromuscular modeling.
- High-dimensional multi-joint personalization: Simultaneously optimizes knee and ankle phase-dependent stiffness, damping, and equilibrium angle without dimensionality reduction.
- Onboard-sensor-only reward: Biomimicry reward computed from prosthesis onboard sensors, no external motion capture needed.
- Strong sim-to-hardware predictive validity: Pearson r=0.96-0.997, best hardware policy in sim Top-5.
5.
Joint torque
$$ K_{j}(s_{\mathrm{st}})\,\big(\theta_{j}(t)-\theta^{\mathrm{eq}}_{j}(s_{\mathrm{st}})\big) $$
Polynomial trajectory
$$ p_{i}(s_{\mathrm{st}})=\sum_{k=0}^{3}w_{i,k}\,s_{\mathrm{st}}^{k} $$
Normalized stride phase
$$ s_{\mathrm{st}}(t)=s(t)/\bar{s}_{\mathrm{TO}} $$
Weighted blending
$$ w_{i}=\alpha_{i}w_{i,\mathrm{data}}+k_{p,i}(\phi_{\mathrm{ref}}-\phi_{\mathrm{sim}}) $$
| Parameter | Value | Description |
|---|---|---|
| Learning rate | 1e-4 | Adam optimizer |
| Batch size | 256 | Training batch |
| Discount $\gamma$ | 0.99 | Reward discount |
| Epochs | 5000 | Training iterations |
Figure 2: Additional results from the paper.
Limitations and Future WorkLimitations:
- Only 3 participants, limited sample size.
- Only validated for level-ground walking, not stairs or ramps.
- Replay constraint assumes hip kinematics unchanged post-personalization, which may not be fully accurate.
Future work: Framework extensible to higher-dimensional parameterizations (e.g., neural-network controllers) and more gait modes.
6. Conclusion
The core idea is using replay-constrained simulation to bypass the human modeling challenge — MuJoCo replays individual walking data's hip kinematics and ground reaction forces, simulating prosthesis knee-ankle dynamics, enabling DRL to train personalized policies in high-dimensional parameter space. The biomimicry reward computed solely from onboard sensors ensures practicality. In 3 amputee participants, biomimicry rewards improved by 42-59%, sim-to-hardware Pearson r reached 0.96-0.997, with best hardware policy consistently predicted within simulation Top-5.
SOURCE LINKS
