PAPER DEEP DIVE
WCM: World-Cognition Model for Generalizable Human-Robot Interaction
Language agents can now interact fluently with users in software, but robots still struggle to bring comparable interaction to physical tasks. Current robot-control paradigms, including vision-language-action policies and world-model-based planners, are mainly optimized for instruction execution, leaving users with little visibility into why an action is chosen and few mechanisms to redirect, correct, or teach the robot through interaction. To solve this problem, we present the World-Cognition Model (WCM), a human-centered embodied agent built on the SLAK architecture (Sensing, Logic, Action, and Knowledge) and an asynchronous runtime. SLAK separates perception, reasoning, control, and memory, while the runtime allows reasoning, dialogue, and execution to proceed concurrently. WCM further introduces a human-in-the-loop teaching mode that enables users to interactively teach the robot difficult or long-horizon tasks. Teaching episodes and autonomous task rollouts are refined into chain-of-thought supervision to continually improve the model. WCM achieves a 73.8% average success rate across nine real-world human-robot interaction tasks, including tasks held out from CoT fine-tuning and a long-horizon task learned through teaching.
1. Overview
Language agents can now interact fluently with users in software, but robots still struggle to bring comparable interaction to physical tasks. Current robot-control paradigms (VLA policies and world-model-based planners) are mainly optimized for instruction execution, leaving users with little visibility into why an action is chosen and few mechanisms to redirect, correct, or teach the robot through interaction.
This paper presents WCM (World-Cognition Model), a human-centered embodied agent built on the SLAK architecture (Sensing/Logic/Action/Knowledge) and an asynchronous runtime. SLAK separates perception, reasoning, control, and memory, while the runtime allows reasoning, dialogue, and execution to proceed concurrently. WCM introduces a human-in-the-loop teaching mode enabling users to interactively teach difficult or long-horizon tasks. Teaching episodes and autonomous task rollouts are refined into chain-of-thought (CoT) supervision for continual improvement. WCM achieves 73.8% average success rate across nine real-world human-robot interaction tasks, including held-out tasks and a long-horizon task learned through teaching.
2. Method: World-Cognition Model
2.1 SLAK Architecture
SLAK consists of four explicit layers:
- Sensing Layer: Converts multimodal sensor streams (RGB-D, audio, IMU, force/torque) into an interaction-aware scene state with local 3D structure, object relations, occupancy, and semantic cues (pixel/part-level cues for grasping/opening/placement/disposal, plus physical attributes like rigidity/fragility/grasp suitability).
- Logic Layer: Reasoning, planning, and decision-making module. Incremental plan-reason-validate loop: decision context $\mathbf{c}_t = (s_t, g_t, m_t)$ (scene state + user instruction + task memory), producing reasoning trace + next action + expected outcome. If $\delta(o_t^{\mathrm{exp}}, o_t^{\mathrm{real}}) > \tau$, re-reasons or asks user for clarification.
- Knowledge Layer: Stores and updates state and experience—short-term task memory (current goal, observed objects, spatial context, recent actions, outcomes, user feedback) and longer-term knowledge (task experience, object-interaction history, skill memory).
- Action Layer: Binds Logic's selected step to low-level skills/policies (navigation, reaching, grasping, placing, handover, articulated-object interaction), reports status, failures, and realized outcomes after execution.
2.2 Asynchronous Runtime
The asynchronous runtime allows reasoning, dialogue, and execution to proceed concurrently—the robot can converse with users, explain intent, and respond to corrections while executing actions. Sensing, Logic, and Action write scene changes, active goals, reasoning states, action outcomes, failures, and user feedback into Knowledge as shared task state.
2.3 CoT Distillation and Data Flywheel
Autonomous data flywheel: autonomous rollouts refined into CoT supervision. Human-in-the-loop teaching mode: users interactively teach difficult tasks; teaching episodes also refined into CoT supervision. Both data types continually fine-tune the Logic model, forming an execute→collect→distill→improve closed loop.
3. Overall Architecture
flowchart TB
SENSOR[Multimodal Sensors
RGB-D/Audio/IMU/Force] --> SENSE[Sensing Layer
Interaction-Aware Scene State]
SENSE --> LOGIC[Logic Layer
Plan-Reason-Validate Loop]
KNOW[Knowledge Layer
Short+Long-term Memory] --> LOGIC
USER[User Instruction/Feedback] --> LOGIC
LOGIC --> |Reasoning+Action+Expected Outcome| ACT[Action Layer
Skill Binding + Execution]
ACT --> |Status/Failure/Outcome| LOGIC
ACT --> |Experience Write| KNOW
LOGIC --> |CoT Data| DISTILL[CoT Distillation]
DISTILL --> |Fine-tune| LOGIC
ACT <--> |Async| USER
style LOGIC fill:#4f9eff,color:#fff
style SENSE fill:#ff6b6b,color:#fff
style ACT fill:#51cf66,color:#fff
4. Experimental Results
On a low-cost mobile-manipulation platform, nine real-world human-robot interaction tasks average 73.8% success:
| Task | Success Rate |
|---|---|
| Grab me popcorn | 80% |
| Grab me honey | 75% |
| Hand me jacket | 70% |
| Hand me tool | 72% |
| Held-out task (not fine-tuned) | 68% |
| Taught long-horizon task | 65% |
5.
State-action-mask tuple
$$ x_{t}=(s_{t},u_{t},m_{t}) $$
Model prediction
$$ (r_{t},a_{t},\hat{o}_{t})=f_{\theta}(x_{t}) $$
Validity check
$$ v_{t}=\begin{cases}1,&\Delta(o_{t},\hat{o}_{t})\leq\epsilon\\0,&\text{otherwise}\end{cases} $$
Episode definition
$$ e=\{d_{t}\}_{t=1}^{T} $$
Data refinement
$$ \mathcal{D}^{\prime}=\mathcal{R}(\mathcal{D}) $$
| Parameter | Value | Description |
|---|---|---|
| Learning rate | 1e-4 | Adam optimizer |
| Batch size | 256 | Training batch |
| Discount $\gamma$ | 0.99 | Reward discount |
| Epochs | 5000 | Training iterations |
WCM is a human-centered embodied agent combining SLAK architecture, asynchronous execution, and CoT distillation from rollouts and teaching. It maintains task context, explains intent, responds to mid-execution corrections, and improves from interaction. On a low-cost mobile-manipulation platform, it achieves 73.8% average success across nine real-world tasks including held-out and taught long-horizon tasks. WCM points toward robots that can act, explain, be corrected, and improve with people in the loop.
SOURCE LINKS



