PAPER DEEP DIVE
Try Once, Then Optimal: De-Redundified Procedure Memory for Cross-Episode Exploration Amortization
Manipulating objects with hidden internal state, such as a latched microwave, forces a robot to probe before it can act. Yet a robot that has solved an instance once re-runs the same probes whenever it encounters that instance again, because existing cross-episode memories target task success and organize reuse around states, not the object or the cost of re-exploring it. We present Instance-Oriented Memory (IOM), an object-centric framework that amortizes this exploration: from a single encounter that uncovers the hidden state, whether or not it succeeds, IOM records a short procedure for manipulating that instance, keys it on the object's identifiable features, and injects it as a soft bias on a procedure-conditioned policy. A later encounter recognizes the object and recalls its procedure instead of re-exploring. We instantiate this distillation with an off-the-shelf vision-language model (VLM) that parses each encounter into the procedure without task-specific training. Across four articulated-object tasks, two in simulation (microwave, door) and two on a real robot (bottle, cabinet), an oracle procedure memory cuts manipulation operations by 16-30% over re-exploration at non-regressing success, and the VLM instantiation recovers 69-88% of that saving out of the box. Because the procedure is a soft bias on a feedback-driven policy, an incorrect memory is recovered from rather than obeyed: success holds even when a retrieved procedure is wrong, as for $\approx$12% of door instances. Across all tasks the benefit is purely one of efficiency: success never regresses, and on the real robot even improves. Code will be released upon acceptance.
Try Once, Then Optimal: De-Redundified Procedure Memory for Cross-Episode Exploration Amortization
Authors: Haizhou Ge, Haochen Ouyang, Zhixing Chen, Yufei Jia, Yue Li, Lu Shi, Lei Han, Guyue Zhou, Ruqi Huang | Institution: Tsinghua University & DISCOVER Robotics | arXiv: 2607.23702v1
One-Sentence Summary
This paper proposes IOM (Instance-Oriented Memory), an object-centric memory framework that distills a de-redundified procedure from a single state-revealing encounter, keys it on the object's identifiable features, and recalls it on later encounters instead of re-exploring — cutting operations by 16–30% across four articulated-object tasks at non-regressing success, with an off-the-shelf VLM recovering 69–88% of the oracle benefit with no training.
Background and Motivation
Many everyday objects carry a hidden internal state (e.g., a microwave's latch) that a robot cannot observe before touching. Manipulating such objects requires probing — interacting to reveal the state before choosing the right action. Probing is unavoidable the first time but pure overhead every time after. A robot that has already opened a particular microwave should not cautiously release a latch it already knows is free.
Figure 1: First encounter probes hidden state → distills de-redundified procedure → writes to memory keyed by $\phi(x)$ → later encounter recognizes object and recalls procedure directly.
Existing cross-episode memories target task success and organize around states, not objects or the cost of re-exploring. IOM takes an object-centric view: a robot should remember the objects it manipulates and how to manipulate them.
Problem Formulation
Consider object class $\mathcal{C}$ and task $\tau$. Let $\mathcal{P}$ be all sequences completing $\tau$, $\mathcal{R} \subseteq \mathcal{P}$ the no-retry sequences. Instance $x \in \mathcal{C}$ has hidden state $s(x)$. A no-retry sequence is non-redundant for $x$ when no operation can be dropped:
$$\mathcal{K}^{\star}(x)=\kappa\bigl(s(x)\bigr)\subseteq\mathcal{K}$$where $\mathcal{K}=\{K_1,\dots,K_m\} \subseteq \mathcal{R}$ is the finite set of de-redundified procedures. Each encounter executes $p \in \mathcal{P}$ with cost $c(p)$ = number of redundant operations:
$$c(p) = 0 \iff p \text{ is non-redundant}$$Uncovering sequences $\mathcal{U}$ determine $s(x)$ through interaction. A memoryless agent re-probes every encounter. IOM discovers $s(x)$ once and reuses it: later encounters execute $K \in \mathcal{K}^\star(x)$ with $c(p_i)=0, \forall i>0$.
Method: Three Modules
1. Procedure Distillation
Recovers a compact procedure from the first encounter's possibly redundant sequence:
$$\mathcal{D}:(\xi,g)\mapsto\hat{K}\in\mathcal{K}$$where $\xi$ is the encounter record and $g$ is the shared task prior. Distillation is governed by revelation, not completion — a failed encounter that still reveals $s(x)$ yields the same $\hat{K}$. Instantiated with a pretrained VLM: the encounter is presented as RGB video + action trajectory, $g$ as a language prompt, and the VLM returns $\hat{K}$ with no task-specific training.
2. Instance-Keyed Memory
A partial map $\mathcal{M}$ from identifiable features $\phi(x)$ to stored procedures:
$$\mathcal{M}\bigl(\phi(x)\bigr)=\begin{cases}\hat{K},&\text{if }x\text{ has been written}\\u,&\text{otherwise}\end{cases}$$where $u \in \mathcal{U}$ is an uncovering probe. Memory is separate from the policy and shareable across tasks. Implemented with a pretrained VLM: writing associates the object's first frame with $\hat{K}$; recall checks if the object has been seen before.
3. Procedure-Conditioned Policy
The policy must accept $K$ as conditioning and be history-dependent (multi-modal actions). It maps:
$$a_{t:t+H}\sim\pi\bigl(\cdot\mid o_t,\,h_t,\,K,\,z\bigr),\qquad K\in\mathcal{K}\cup\mathcal{U}$$Adapted from AdaManip: diffusion policy replaced with flow matching, language-conditioning channel added for $K$. The prompt $K$ is embedded by a pretrained text encoder and concatenated with observation features.
Soft-Bias Conditioning Training
For executed sequence $p$ in encounter $\xi$, collect conditions $\mathcal{K}^\circ(p)$ that open with $p$'s pre-retry prefix but are otherwise unconstrained. Training samples $K$ randomly from this set, teaching the policy to follow $K$ at the opening yet recover through feedback when execution diverges.
flowchart TD
A["First encounter: probe u ∈ U"] --> B["Reveals hidden state s(x)"]
B --> C["Distillation D: (ξ,g) → K̂"]
C --> D["Write memory: M[φ(x)] ← K̂"]
D --> E["Later encounter: identify φ(x)"]
E --> F{"In memory?"}
F -- "Hit" --> G["Recall K̂ as soft bias"]
F -- "Miss" --> A
G --> H["Policy π(·|o_t, h_t, K, z)"]
H --> I["Execute: correct → direct completion
wrong → recover via feedback"]
Experimental Results
Evaluated on two simulation tasks (microwave, door) and two real-robot tasks (bottle, cabinet). Simulation: 90 instances (50+40), 5 episodes each. Real robot: AIRBOT Play arm + Azure Kinect DK, 5 trials each from locked/unlocked starts. Variants: Random (no memory), GT (oracle), VLM (GPT-5.5 generated).
| Task | Memory | Ops ↓ | vs Random | Success ↑ |
|---|---|---|---|---|
| Microwave (sim) | Random | 1.96 | — | 1.000 |
| GT | 1.37 | -30.2% | 0.980 | |
| VLM | 1.44 | -26.5% | 1.000 | |
| Door (sim) | Random | 2.54 | — | 0.975 |
| GT | 2.13 | -16.2% | 0.975 | |
| VLM | 2.26 | -11.1% | 0.975 | |
| Bottle (real) | Random | 2.12 | — | 0.80 |
| GT | 1.56 | -26.8% | 0.90 | |
| VLM | 1.67 | -21.6% | 0.90 | |
| Cabinet (real) | Random | 1.86 | — | 0.70 |
| GT | 1.56 | -16.2% | 0.90 | |
| VLM | 1.62 | -12.5% | 0.80 |
Figure 2: Four articulated-object tasks with operation sequences. Hidden states (latch, handle direction, cap tightness) are unobservable from the initial view.
Key Findings
- Framework value: Oracle memory cuts 16–30% operations across all 4 tasks at non-regressing success (real-robot success even improves).
- Training-free instantiation: VLM recovers 69–88% of oracle benefit with no task-specific training.
- Robustness: ~12% of door instances receive wrong procedures, yet success is unchanged — soft bias on feedback-driven policy degrades gracefully.
- Concentrated savings: Excluding the first memory-free encounter, oracle per-repeat reduction rises to 20–33% in simulation.
- Recall over fine perception: Bottle cap tightness shows only as a narrow gap, too fine for reliable image reading; IOM resolves state through action once and recalls by instance key thereafter.
| Task | Non-redundant $\mathcal{K}$ | Probe $\mathcal{U}$ | With retry |
|---|---|---|---|
| Microwave | [open], [release, open] | [open] | [open, release, open] |
| Door | [cw, open], [ccw, open] | [cw, open] | [cw, ccw, open] |
| Bottle | [lift], [unscrew, lift] | [lift] | [lift, unscrew, lift] |
| Cabinet | [pull], [release, pull] | [pull] | [pull, release, pull] |
Figure 3: Atomic operation sequences and hidden state illustration for the four tasks.
Limitations
- Memory key $\phi(x)$ is appearance-based; perceptual aliasing occurs when different objects look similar but require different procedures.
- No self-repair of incorrect memory entries — requires meta-memory monitoring recall outcomes.
- Only tested the reset setting (hidden state resets between encounters); persistent settings with history-dependent optimal procedures are left to future work.
Conclusion and Outlook
IOM is a modular, object-centric framework that amortizes manipulation exploration across repeated encounters by recording a de-redundified procedure from a single uncovering encounter and recalling it on later encounters. Three interchangeable modules (procedure distillation, instance-keyed memory, procedure-conditioned policy) are realized with off-the-shelf components. Across four articulated-object tasks, IOM reduces operations by 16–30% at non-regressing success.
Key insight: "Try once, then optimal." — Converting brittle online perceptual judgments into efficient lookups after a single revealing encounter is IOM's core value proposition. Future work will explore more discriminative memory keys (extending to scene context) and self-correcting meta-memory.
Deep analysis generated by RobotWorld paper-detail-generator based on full-text reading | arXiv:2607.23702v1
SOURCE LINKS
