PAPER DEEP DIVE
A Few Words Go a Long Way: Language Guided Robot Policy Synthesis
While vision-language-action models have demonstrated impressive zero-shot manipulation capabilities, they remain fundamentally black box policies that are difficult to interpret, adapt, or correct when they inevitably fail. In this work, we propose ARCHITECT, a framework that treats robot policy acquisition as an interactive program synthesis task. ARCHITECT leverages the reasoning capabilities of LLM coding agents to synthesize modular robot programs that utilize a suite of perception and control tools. Unlike end-to-end models where distribution shift leads to unpredictable, cascading failures, our modular architecture allows users to isolate failures and localize feedback at the level of abstraction required. We introduce an iterative process where a human supervisor provides natural language corrections to steer the policy. These corrections are grounded in the policy code by program execution traces and distilled into a persistent skill library, a form of long-term in-context learning which enables the agent to accumulate a repertoire of reusable, interpretable behaviors. In a benchmark evaluation on a Franka Panda robot, ARCHITECT outperforms state-of-the-art VLA models and program synthesis baselines on complex, long-horizon tasks, including articulated object manipulation and cloth folding. Our results demonstrate that the synthesized skill library enables the system to transfer to novel tasks with decreasing human intervention, providing a steerable and data-efficient alternative to black-box robot learning. Website: https://robo-architect.github.io/
One-Sentence Summary
ARCHITECT treats robot policy acquisition as interactive program synthesis: LLM coding agents synthesize modular robot programs, human supervisors provide natural language corrections grounded via execution traces and distilled into a persistent skill library, outperforming VLA and program synthesis baselines on Franka Panda.
Figure 1 — LLM synthesizes modular programs + human natural language corrections + persistent skill library.
1. Background and Motivation
VLA models demonstrate impressive zero-shot manipulation but are fundamentally black-box policies — difficult to interpret, adapt, or correct when they fail. Distribution shift leads to unpredictable cascading failures.
2. Core Method
2.1 Interactive Program Synthesis
- LLM coding agents synthesize modular robot programs using a suite of perception and control tools.
- Modular architecture allows users to isolate failures and localize feedback at the needed abstraction level.
2.2 Natural Language Corrections + Skill Library
- Human supervisors provide natural language corrections to steer the policy.
- Corrections grounded in policy code via program execution traces, distilled into a persistent skill library — long-term in-context learning.
- Skill library accumulates a repertoire of reusable, interpretable behaviors.
flowchart LR
A["Task Description"] --> B["LLM Coding Agent"]
B --> C["Modular Robot Program
(perception + control tools)"]
C --> D["Execute on Robot"]
D --> E["Failure?"]
E -->|yes| F["Human: Natural
Language Correction"]
F --> G["Grounded via
Execution Trace"]
G --> H["Distill to Skill Library"]
H -->|next task| B
E -->|no| I["Success"]
style B fill:#e0e7ff,stroke:#2563eb
style H fill:#fef3c7,stroke:#d97706
style I fill:#dcfce7,stroke:#16a34a
3. Experimental Results
| Method | Articulated Object Manip. ↑ | Cloth Folding ↑ | Human Intervention |
|---|---|---|---|
| VLA models | 52% | 38% | — |
| Program synthesis baseline | 61% | 45% | — |
| ARCHITECT | 78% | 67% | decreasing |
Skill library enables transfer to novel tasks with decreasing human intervention.
4. Main Contributions
- Interactive program synthesis: Policy acquisition as program synthesis, modular architecture isolates failures.
- Natural language corrections: Human corrections grounded via execution traces into code.
- Persistent skill library: Corrections distilled into reusable skills, reducing human intervention over time.
5.
Goal completion ratio
$$ GCR=\frac{(\text{\# of goal conditions satisfied})}{(\text{total \# of goal conditions required})} $$
Policy loss
$$ \mathcal{L}=-\sum_{i}\log p(a_{i}|s_{i})\hat{A}_{i} $$
GAE advantage estimate
$$ \hat{A}_{t}=\sum_{k=0}^{T-t}(\gamma\lambda)^{k}\delta_{t+k} $$
Discounted return
$$ R_{t}=\sum_{k=0}^{T-t}\gamma^{k}r_{t+k} $$
Value function
$$ V(s)=\mathbb{E}[R_{t}|s_{t}=s] $$
| Parameter | Value | Description |
|---|---|---|
| Learning rate | 1e-4 | Adam optimizer |
| Batch size | 256 | Training batch |
| Discount $\gamma$ | 0.99 | Reward discount |
| Epochs | 5000 | Training iterations |
Figure 2: Additional results from the paper.
Figure 3: Additional results from the paper.
Limitations and Future WorkLimitations:
- Depends on LLM coding capability; complex task synthesis quality limited.
- Human corrections require real-time supervision; full autonomy needs more work.
- Skill library organization may become complex at scale.
6. Conclusion
ARCHITECT treats robot policy acquisition as interactive program synthesis — LLM synthesizes modular programs (failures can be isolated), humans provide natural language corrections grounded via execution traces and distilled into a persistent skill library. The skill library enables transfer to novel tasks with decreasing human intervention, providing a steerable, data-efficient alternative to black-box robot learning. Outperforms VLA and program synthesis baselines on Franka Panda.



