ADEPT: Pre-Train Dexterity Once, Post-Train Every Task

Teaching a multi-fingered robot hand to do anything useful has traditionally meant training a fresh reinforcement-learning policy from scratch for every single task. Each new task rediscovers reaching, grasping, lifting, and in-hand reorientation on its own, burning billions of simulated steps and still failing more often than not. A new paper from NVIDIA and the University of Michigan argues the economics of dexterity are wrong, and proposes to fix them the same way large language models fixed language: pre-train once, then adapt cheaply.

ADEPT (Accelerating Dexterity via Pre-Training) pre-trains a single dexterous policy on a generic object-reposing task in simulation, post-trains it into task-specific specialists without erasing what it already knows, and distills the result into perception-based student policies that deploy zero-shot on real hardware. On a 23-DoF Kuka iiwa + Allegro hand running on two RGB cameras, and a 29-DoF Flexiv Rizon + Sharpa hand running on RGB plus five vision-based tactile sensors, ADEPT policies solve long-horizon pick–reorient–insert tasks end-to-end at human-comparable speed (5–10 s per trial), with no demonstrations, no scripted stages, and no external fixtures.

Why Dexterity Keeps Starting From Zero

Arm–hand systems combine high-dimensional state and action spaces with contact-rich dynamics. Useful behaviors are sparse islands in a huge space, and task-specific rewards give RL little guidance about how to reach them. Even with massively parallel GPU simulation, a policy trained from scratch on one dexterous task rarely transfers to another — every new task pays the full price again.

ADEPT's answer is to separate what is shared from what is task-specific. Dexterous robots repeatedly need the same core abilities — reach, grasp, lift, reorient, transport — regardless of the downstream objective. So ADEPT learns that shared foundation once, on a generic reposing task, and treats every downstream problem as a cheap adaptation on top of it. In the paper's numbers, pre-training costs about 8B environment steps but is amortized across all downstream tasks; each new task then post-trains in roughly 3B steps, whereas training the same task from scratch takes around 9B steps and is highly seed-sensitive, with most seeds never succeeding.

Learn dexterity once. Build every new behavior on top.

The ADEPT Recipe

Everything is formulated as a discrete-time MDP with action space $\mathcal{A}=[-1,1]^{n_q}$ ($n_q=23$ for Kuka–Allegro), optimizing the discounted return

$$J(\theta)=\mathbb{E}_{\tau\sim\pi_{\theta}}\left[\sum_{t=0}^{T}\gamma^{t}r(s_{t},a_{t})\right]$$

via PPO with an asymmetric actor-critic. ADEPT operates over two MDPs, $\mathcal{M}_{\mathrm{pre}}$ and $\mathcal{M}_{\mathrm{post}}$, which share the action space but differ in observations, dynamics, and reward: the downstream observation $o^{\mathrm{post}}$ extends $o^{\mathrm{pre}}$ with task-specific signals such as the receptacle pose and object–receptacle contact forces.

flowchart LR
    P["Stage 1: Pre-train\nGeneric object reposing\n16 primitives, ADR, PBT"] --> PI1["pretrained prior\npi_pre, V_pre"]
    PI1 --> PT["Stage 2: Post-train\nBC distillation\ncritic warm-up\nconservative PPO"]
    PT --> PI2["task specialist\npi_post, V_post"]
    PI2 --> D["Stage 3: Distill\nDAgger + 8-keypoint aux loss\ntwo-stage vision curriculum"]
    D --> S["student policy\nRGB / visuo-tactile"]
    S --> R["Stage 4: Deploy zero-shot\nKuka-Allegro 23 DoF\nFlexiv-Sharpa 29 DoF"]

Stage 1 — Pre-training foundational dexterity

The pre-training curriculum is strikingly simple in scope: at each episode, one of 16 primitive shapes (cylinders, cuboids, spheres, cones) at a randomized scale spawns on the table, and the policy must execute the full sequence of reaching, grasping, lifting, in-hand reorientation, transporting, and reposing the object onto a sampled target pose. That single objective, randomized hard enough, turns out to induce the entire reusable dexterous repertoire.

Three ingredients carry the load. First, Automatic Domain Randomization (ADR) acts as an online curriculum: task goals advance and environmental complexity increases only as the agent's success rate justifies it, with gravity itself annealed from 0 to $-9.81\ \mathrm{m/s^2}$. Second, Population-Based Training (PBT) runs $N=16$ policies in parallel, periodically replacing the bottom 40% of the population with mutated copies of top performers' weights while preserving each policy's own ADR level — an exploited policy resumes against an easier curriculum slice with a donor's well-shaped weights. Third, objects are represented as point clouds, which the authors found gives better zero-shot generalization to downstream tasks. The result is a reusable prior $\pi_{\mathrm{pre}}$ whose emergence of natural grasps, finger coordination, and in-hand reorientation is analyzed later in the paper.

Stage 2 — Post-training without forgetting

Here is where naive transfer breaks. The pretrained policy can zero-shot the reposing segment of a downstream task it has never seen — yet standard PPO fine-tuning destroys that capability almost immediately. The paper traces the collapse to three mismatches: a changed reward function, extra observations only available downstream (receptacle pose $\mathbf{p}_{\mathrm{rec}}$, contact forces $\mathbf{f}_{\mathrm{or}}$), and therefore unreliable value estimates that feed oversized policy updates. The pretrained critic $V_{\mathrm{pre}}$ is calibrated to the reposing reward; under the insertion reward its advantages are wrong, the actor is pushed off its pretrained manifold faster than the critic can recalibrate, and within a few iterations the behavior is gone (success rate collapses to 0%).

ADEPT's fix is a three-step structured recipe, framed as layering new behavior on top of the prior rather than relearning manipulation:

1. BC actor distillation. Distill $\pi_{\mathrm{pre}}$ into a new downstream actor $\pi_{\mathrm{post}}$ that accepts the expanded observation space, via supervised imitation for 40k iterations.
2. Critic warm-up. Freeze $\pi_{\mathrm{post}}$ and train a fresh critic $V_{\mathrm{post}}$ on its rollouts under the downstream reward for 20 PPO iterations ($\sim$1M env steps per GPU with 4096 envs), so value estimates are calibrated before any policy update happens.
3. Conservative PPO. Unfreeze and jointly update $(\pi_{\mathrm{post}}, V_{\mathrm{post}})$ with a decaying actor learning rate ($10^{-3}\to10^{-5}$), tightened PPO clip ($\epsilon: 0.2\to0.05$), and fixed critic LR $5\times10^{-5}$.

Stage 3 — Distilling specialists into perceptive students

The post-trained teacher consumes privileged simulation state, so it cannot run on a real robot. ADEPT distills it with DAgger into a student that sees only proprioception, fabric state, two RGB images — and, on Flexiv–Sharpa, five per-finger tactile depth maps. The distillation objective combines a Mahalanobis-weighted behavior-cloning term, where the teacher's per-joint standard deviation acts as precision weighting,

$$\mathcal{L}_{\mathrm{BC}}=\sqrt{\sum_{i=1}^{n_{q}}\frac{\bigl(\mu_{\theta}^{i}(o_{t})-\mu_{T}^{i}(o_{t}^{T})\bigr)^{2}}{(\sigma_{T}^{i})^{2}}}\;+\;\sqrt{\sum_{i=1}^{n_{q}}\frac{\bigl(\sigma_{\theta}^{i}(o_{t})-\sigma_{T}^{i}\bigr)^{2}}{(\sigma_{T}^{i})^{2}}}$$

with an auxiliary 8-keypoint object pose loss that forces the visual encoder to actually perceive the peg's orientation — the dominant failure mode of insertion:

$$\mathcal{L}_{\mathrm{aux}}=\sqrt{\sum_{i=1}^{8}\bigl\|\hat{\mathbf{k}}^{i}_{\theta}(o_{t})-\mathbf{k}^{\star,i}_{t}\bigr\|_{2}^{2}},\qquad \mathcal{L}_{\mathrm{distill}}=\mathbb{E}_{o_{t}\sim\pi_{\theta}}\left[w_{\mathrm{BC}}\,\mathcal{L}_{\mathrm{BC}}+w_{\mathrm{aux}}\,\mathcal{L}_{\mathrm{aux}}\right]$$

with $w_{\mathrm{BC}}=1$ and $w_{\mathrm{aux}}=20$. Mirroring the RL pipeline, the student itself follows a two-stage curriculum: first it is distilled from the reposing teacher on a perception-heavy surrogate task (detect the peg, track its 8-keypoint pose through grasp and lift, reproduce the reorientation), and only then from the downstream teacher on the contact-rich insertion task. Stage 1 lets $\mathcal{L}_{\mathrm{aux}}$ shape the visual encoder alone; stage 2 lets $\mathcal{L}_{\mathrm{BC}}$ refine contact-rich behavior on top of an already-competent encoder. A single-stage baseline fails to transfer entirely (0/10 in the real world), while the two-stage curriculum beats it by roughly 10% in instantaneous success throughout training.

For the visuo-tactile Flexiv–Sharpa student, each fingertip's vision-based tactile sensor is simulated in training as a geometry-consistent penetration-depth map (TacMap), identical in representation to the calibrated real sensor. Depth maps are thresholded into binary contact maps, both channels pass through a per-finger CNN, and the resulting features are spatially anchored via FiLM conditioning on the fingertip position from forward kinematics — giving the policy a spatially grounded view of hand–object contact without any extra sim-to-real bridge.

The safety layer: a full-Cspace geometric fabric

Between the policy and the robot sits a joint-space geometric fabric, an autonomous second-order system on configuration space:

$$\mathbf{M}_{f}(\mathbf{q}_{f},\dot{\mathbf{q}}_{f})\,\ddot{\mathbf{q}}_{f}+\mathbf{f}_{f}(\mathbf{q}_{f},\dot{\mathbf{q}}_{f})+\mathbf{f}_{\pi}(\mathbf{a})=\mathbf{0}$$

The policy emits per-joint relative deltas $\mathbf{a}_{t}\in[-1,1]^{n_q}$ which the fabric consumes as a Cspace target, while its autonomous terms $\mathbf{f}_f$ enforce joint limits, self- and environment-collision avoidance, damping, and speed control. Crucially, the fabric operates in the full joint configuration space — exposing the complete 23-DoF kinematic dexterity to the policy, unlike prior fabric-guided dexterous policies that restrict the hand to a low-dimensional PCA grasp subspace — and the identical fabric instance runs in simulation and on hardware, keeping the controller gap near zero.

Results: Generalization, Efficiency, and Zero-Shot Real Deployment

The prior generalizes to objects it never saw. Evaluated on the reposing task, the Kuka–Allegro teacher matches or slightly exceeds its in-distribution success rate on two out-of-distribution sets — FMB pegs and 152 VisDex objects — despite training only on 16 primitives.

Robot	16 primitives (in-dist.)	FMB pegs	VisDex (152 objects)
Kuka–Allegro	0.73 ± 0.003	0.76 ± 0.003	0.77 ± 0.011
Flexiv–Sharpa	0.64 ± 0.007	0.58 ± 0.011	0.61 ± 0.015

The prior zero-shots the reposing segment of downstream tasks. On FMB peg reposing across ADR levels, the pretrained teacher maintains above 50% success through ADR level 35, and only drops to ~0% at the actual insertion goal (ADR 50), where contact with the receptacle begins — exactly the segment post-training is meant to learn. All downstream post-training therefore starts at ADR 20, where the prior is still reliable.

Post-training is cheaper and more reliable than training from scratch. ADEPT reaches a downstream teacher in 3B steps on top of the amortized 8B-step pretraining. Training from scratch needs ~9B steps when it works at all — most seeds plateau early and never reach the final ADR level. Ablations over five seeds per variant show the reduced actor learning rate is the component that prevents collapse (every $10^{-3}$ variant collapses, even with BC and warm-up, and even with a KL penalty); critic warm-up adds +17.6 points of success rate; and BC nearly halves adaptation time (19.9 h vs 35.2 h to ADR 50).

Zero-shot on hardware. A single Kuka–Allegro reposing checkpoint post-trains into three downstream specialists — star-peg insertion, square-and-round peg insertion, and dish-rack placement — with no task-specific pre-training. Students deploy zero-shot, with no real-world fine-tuning. The visuo-tactile Flexiv–Sharpa student reaches 8/10 on the hardest FMB peg; vision-only students manage 5/10 (star), 3/10 (square/round) on Kuka–Allegro and 6/10 on dish placement. Each stage below is cumulative — success at a stage requires success at all earlier stages.

Modality	Robot	Task	Reach	Grasp	Lift	Reorient	Align	Insert (SR)
Vision	Kuka–Allegro	FMB Star	10/10	9/10	8/10	8/10	7/10	5/10
Vision	Kuka–Allegro	FMB Square/Round	10/10	8/10	6/10	4/10	3/10	3/10
Vision	Flexiv–Sharpa	FMB Square/Round	10/10	7/10	5/10	3/10	3/10	3/10
Visuo-Tactile	Flexiv–Sharpa	FMB Square/Round	10/10	10/10	10/10	9/10	8/10	8/10
Vision	Kuka–Allegro	Dish Placement	10/10	10/10	8/10	7/10	6/10	6/10

The tactile result is the starkest finding: in real deployment, touch is decisive (8/10 vs 3/10 overall on Flexiv–Sharpa for the same task). The vision-only student's failure mode is not execution but confidence — it cannot tell whether a grasp succeeded, so it reopens the hand, drops the object, and loops between grasping and regrasping. With fingertip tactile maps the policy is certain of its contacts: it grasps and lifts successfully in every trial (10/10) and carries that reliability all the way through insertion.

Emergent Dexterity: Natural Grasps Nobody Programmed

Nothing in the insertion reward constrains how the peg is grasped. From random initialization, PPO happily converges to unnatural grasps, with behavior varying significantly across seeds. What produces natural grasps is the reposing pre-training itself: it initializes the policy in a region of parameter space where natural grasps emerge, and post-training's local updates refine those grasps rather than rediscovering them. Task-misaligned grasps the teacher occasionally produced (e.g., grasping the peg from the bottom) disappear under post-training as guided by the downstream alignment reward.

The dish task shows post-training can do more than polish existing skills. A plate is geometrically far from anything among the 16 primitives, and no pretrained grasp works on it zero-shot. But the prior still provides a useful starting distribution — the policy reaches the plate and produces plausible near-miss grasp attempts — from which downstream RL learns successful plate grasping and, for upside-down plates, a flip-and-regrasp strategy the pretrained policy never exhibited. Pre-training does not need to already solve the downstream behavior; it only needs to place the policy where downstream RL can find the solution.

Capabilities in Motion

Hands vs. Jaw Grippers: 2–14× Faster

The Functional Manipulation Benchmark was originally built for parallel-jaw grippers, whose solution decomposes the task into grasp, place-on-fixture, regrasp, rotate, and insert stages with external fixtures, taking 20–70 s per trial. ADEPT's multi-fingered policies run the entire sequence as one continuous behavior in 5–10 s — a 2×–14× execution-time speedup — while discovering in-hand reorientation strategies that are awkward or impossible to demonstrate with a gripper.

Limitations and What Comes Next

The authors are candid about the bottleneck: perception. Real-world failures concentrate on incorrect estimates of the asymmetric peg's orientation under occlusion, and the Allegro's rounded fingertips make a small contact patch against the peg's rounded side during fast arm motion. Fingertip touch mitigates contact ambiguity — the Flexiv–Sharpa numbers prove it — but robust object-centric perception under occlusion remains open; wrist-mounted cameras and tactile sensing on more platforms are the suggested next steps. On the pre-training side, the authors plan to broaden the prior toward in-hand manipulation, tool use, clutter, and bimanual settings, and to measure how far such priors can transfer to tasks increasingly unlike the pre-training distribution.

Why It Matters

ADEPT is one of the clearest demonstrations yet that the pre-train/post-train paradigm — the economic engine behind LLMs — transfers to contact-rich physical skill. The evidence is specific: an amortized pre-training cost, marginal task cost cut from ~9B to ~3B steps, zero-shot real deployment on two different arm–hand embodiments from raw perception alone, and emergent natural grasping as a byproduct of the prior rather than an engineered objective. If reusable dexterous priors keep scaling, "training a new manipulation behavior" may stop meaning "months of RL from scratch" and start meaning what fine-tuning already means in language: hours of adaptation on top of a foundation everyone shares.

Source

This post is based on the paper "ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning" by Jayjun Lee, Jessica Yin, Asif Rana, Nicholas Blauch, Sam Mady, Mohak Bhardwaj, Nima Fazeli, Nathan Ratliff, Karl Van Wyk, and Ankur Handa (NVIDIA; University of Michigan), and on the accompanying project page with videos and interactive demos.

• Paper: arXiv:2608.19182
• Project page: adept-dexterity.github.io