Skip to content
RobotWorld
Back to Papers

PAPER DEEP DIVE

具身智能PaperEmbodied AI

LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratories

The deployment of embodied agents in self-driving laboratories could accelerate scientific discovery, yet their reliability is constrained by the irreversible and safety-critical nature of chemical experiments. Progress is further hindered by scarce failure data and the lack of fine-grained evaluation protocols. To address these challenges, we introduce LabRobFail, a failure-centric framework for learning and evaluating robotic failure analysis in chemical laboratories. LabRobFail-Sim injects controllable failures at the control, physics, and semantic levels, enabling the construction of LabRobFail-Data, which contains over 20,000 trajectories across 70+ task scenarios, five failure categories, and 11 fine-grained failure types. LabRobFail-Bench evaluates six capabilities spanning task understanding, failure detection, temporal localization, severity assessment, failure classification, and actionable correction. We further develop LabRobFail-VLM, a domain-specialized vision-language model that generates structured failure diagnoses and recovery instructions. On seen environments, it achieves 92.58% failure-detection accuracy and 85.58% temporal-localization accuracy, substantially outperforming general-purpose VLMs. When integrated as a real-time supervisor, it improves downstream VLA task success rates by 10-20 percentage points, demonstrating the value of fine-grained failure understanding for closed-loop recovery and reliable laboratory autonomy. Our code and data are available at https://github.com/Su-ISE-2001/SciRobo

Haobo Wang, Baoli Sun, Anqi Zou, Dongsheng Huang, Zelin Lv, Ning Wang, Rui Li, Dongzhan Zhou, Weiyu Guo, Zhihui Wang, Wanli OuyangJuly 26, 20267 min read
中文

LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratory

Paper: LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratory

Authors: Haobo Wang, Baoli Sun, Anqi Zou, Dongsheng Huang, Zelin Lv, Ning Wang, Rui Li, Dongzhan Zhou, Weiyu Guo, Zhihui Wang, Wanli Ouyang

Link: arXiv:2607.23704 | Code: GitHub


One-Sentence Summary

LabRobFail is a failure-centric framework for chemical self-driving laboratories that injects controllable failures at control/physics/semantic levels via LabRobFail-Sim, builds a dataset of 20,000+ trajectories with 11 fine-grained failure types, develops a six-dimensional evaluation benchmark, and a domain-specialized VLM achieving 90.83% failure-detection accuracy and 77.21% temporal-localization accuracy, improving downstream task success rates by 4-16 percentage points as a real-time supervisor.


Background and Motivation

Deploying embodied agents in self-driving laboratories can accelerate scientific discovery, but reliability is constrained by the irreversible and safety-critical nature of chemical experiments. Progress is further hindered by scarce failure data and the lack of fine-grained evaluation protocols. Existing robotic failure detection datasets (RLBench-Fail, AHA, RoboFAC) do not support chemical laboratory environments or specialized instruments, and lack fine-grained correction, temporal localization, and severity assessment capabilities.

The uniqueness of chemical lab scenarios lies in: transparent glassware, reflective liquid surfaces, and irreversible hazardous mixtures, making failure detection more challenging than household scenarios. LabRobFail is designed to fill this gap.

Framework Overview

Figure 1: LabRobFail Framework. A high-fidelity simulation platform automatically injects five failure categories: Perception (PF), Grasping (GF), Motion (MF), Logic (LF), and Safety (SF).


Method Details

1. LabRobFail-Sim: Multi-level Failure Injection

LabRobFail-Sim injects controllable failures at three levels to generate diverse failure trajectories.

Control-level Perturbation ($\Phi_{ctrl}$): Simulates execution errors by injecting stochastic noise into keyframe parameters. Gaussian perturbation on translation $\mathbf{p}_i$ and Lie algebra noise on rotation $\mathbf{R}_i$:

$$ \tilde{p}_i = p_i + \xi_{trans}, \quad \tilde{R}_i = R_i \cdot \text{Exp}(\xi_{rot}) $$

where $\xi \sim \mathcal{N}(0, \Sigma)$. Gripper commands are corrupted with failure probability $\lambda_{grip}$ to model actuator faults.

Physics-level Perturbation ($\Phi_{phy}$): Dynamically adjusts simulation dynamics parameters $\Psi = \{\mu, m, \nu, \dots\}$ (friction, mass, viscosity) via uniform scaling to induce emergent failures like slippage:

$$ \tilde{\psi} = \psi \cdot (1 + \delta), \quad \delta \sim \mathcal{U}(-\alpha, \alpha) $$

Semantic-level Perturbation ($\Phi_{sem}$): Introduces task logic errors modeled as a permutation operator $\pi$ on sequence indices to simulate step reversals or skips ($\tilde{\tau}_{seq} = \{T_{\pi(1)}, \dots\}$), and a mapping function $\mathcal{M}(o_i)$ for target object substitution (e.g., wrong reagent), leading to safety constraint violations.

Sim Framework

Figure 2: LabRobFail-Sim Framework. (a) Multi-level failure generator; (b) LLM-driven semantic annotation pipeline.

2. LabRobFail-Data Dataset

A large-scale dataset with 20,000+ interaction trajectories across 70+ laboratory task scenarios. Tasks are organized into three complexity levels:

  • Short-horizon ($\leq 10$ frames): Atomic operations (pick, place, pour); failures are typically instantaneous execution errors
  • Medium-horizon (10-30 frames): Chain 2-3 atomic operations, examining failure detection at critical action transitions
  • Long-horizon ($> 50$ frames): Complex multi-stage workflows requiring hierarchical planning and precise fault localization

The failure taxonomy covers 5 major categories and 11 fine-grained types: Perception Failure (PF), Grasping Failure (GF), Motion Failure (MF), Logic Failure (LF), and Safety Failure (SF). Paired success-failure trajectories are generated to minimize confounds.

Dataset and Benchmark

Figure 3: LabRobFail-Data and LabRobFail-Bench. (a) Task complexity distribution; (b) Failure type distribution; (c) Six-dimensional benchmark.

3. LabRobFail-Bench Six-Dimensional Evaluation

The failure analysis task is defined as a mapping from multi-modal observations to a structured diagnostic report. Given input $\mathcal{X} = \langle\mathcal{V}, \mathcal{Q}\rangle$ (RGB video stream $\mathcal{V}$ and task instruction $\mathcal{Q}$), the model predicts a composite failure state $\mathcal{Y}$:

$$ \mathcal{Y} = \langle \underbrace{y_{task}}_{L1},\, \underbrace{y_{dete},\, y_{loca}}_{L2},\, \underbrace{y_{type},\, y_{risk},\, y_{corr}}_{L3} \rangle $$

encompassing task status, detection flags, temporal localization, failure type, risk severity, and actionable correction policies, organized into three progressive cognitive levels.

4. LabRobFail-VLM Vision-Language Model

Built upon Qwen3-VL, consisting of a vision encoder $E_\phi$, modality projector $P_\psi$, and language decoder $D_\theta$. Given input $\mathcal{X} = (X_{\text{grid}}, \mathcal{Q}_{\text{instruct}})$, it extracts visual and textual representations:

$$ Z_v = P_\psi(E_\phi(X_{\text{grid}})), \quad Z_t = \text{Tokenizer}(\mathcal{Q}_{\text{instruct}}) $$

Then autoregressively predicts the failure output:

$$ P(\mathcal{Y} \mid \mathcal{X}) = \prod_{j=1}^{L} D_\theta(y_j \mid y_{<j}, Z_v, Z_t) $$

Hybrid Fine-tuning Strategy: Parameters are divided into vision-related $\Theta_V = \{\phi, \psi\}$ and language-related $\Theta_L = \{\theta\}$. Vision encoder and projector are fully fine-tuned (lr $2 \times 10^{-6}$ and $1 \times 10^{-5}$), while the language decoder uses LoRA (rank 64, scaling 128, dropout 0.05, lr $1 \times 10^{-4}$). Training objective:

$$ \min_{\phi, \psi, \Delta\theta} \sum_{(\mathcal{X}, \mathcal{Y}) \in \mathcal{D}} -\log P(\mathcal{Y} \mid \mathcal{X};\, \phi, \psi, \theta_0 + \Delta\theta) $$ VLM Architecture

Figure 4: LabRobFail-VLM Overview. Temporally indexed multi-view keyframes processed by hybrid-tuned Qwen3-VL.

flowchart TD
    A["Chemical Lab Simulation"] --> B["LabRobFail-Sim
Three-level failure injection"] B --> B1["Control: Gaussian+Lie algebra
Translation/rotation/gripper"] B --> B2["Physics: Dynamics parameter scaling
Friction/mass/viscosity"] B --> B3["Semantic: Step permutation+object substitution
Logic errors+safety violations"] B1 --> C["LabRobFail-Data
20K+ trajectories, 11 failure types"] B2 --> C B3 --> C C --> D["LLM Semantic Annotation
GPT-5.4 VQA pipeline"] D --> E["LabRobFail-Bench
Six-dimensional evaluation"] E --> F["LabRobFail-VLM
Qwen3-VL hybrid fine-tuning"] F --> F1["Vision: Full fine-tuning"] F --> F2["Language: LoRA adaptation"] F1 --> G["Structured failure diagnosis+correction"] F2 --> G G --> H["Downstream policy recovery
+4-16pp success rate"] style B fill:#e1f5fe style F fill:#fff3e0 style G fill:#e8f5e9

Experimental Results

Main Results (Seen Environments)

Table 1: Main results on LabRobFail-Bench (Seen Environments, accuracy %)
MethodQ1 TaskQ2 DetectionQ3 LocalizationQ4 SeverityQ5 ClassificationQ6 ROUGE-L
Qwen3-VL-8B47.4559.505.5947.8435.880.2976
LLaVA-1.6-13B35.8052.9613.2433.5331.180.2598
Gemini-2.5-flash38.9257.1315.5348.8439.680.1231
GPT-5.445.1652.3415.8847.0638.540.1520
LabRobFail-VLM94.4590.8377.2183.1873.210.7542

LabRobFail-VLM dramatically outperforms general-purpose VLMs across all dimensions: failure detection 90.83% (vs GPT-5.4's 52.34%), temporal localization 77.21% (vs 15.88%), correction ROUGE-L 0.7542 (vs 0.1520).

Ablation Study

Table 2: Ablation study (Seen split)
VariantQ2 DetectionQ3 LocalizationQ5 ClassificationQ6 ROUGE-L
Full fine-tuning85.2472.2470.240.7142
Frozen vision81.1360.2161.240.4529
Without temporal index86.4570.5869.240.6471
LabRobFail-VLM (full)90.8377.2173.210.7542

Hybrid fine-tuning outperforms full fine-tuning (Q2: 90.83 vs 85.24). Freezing vision drops Q6 from 0.7542 to 0.4529, proving the critical role of visual full fine-tuning for lab-specific perceptual cues.

Downstream Policy Recovery

Table 3: Downstream policy recovery task success rates
TaskPolicyBaseline(%)+Ours(%)Improvement
PickOpenVLA7684+8%
PourOpenVLA1628+12%
PourACT3248+16%
Open drying ovenOpenVLA2840+12%
Safety Failure Example

Figure 5: Safety failure example — protocol violation (hazardous mixture).


Limitations

  1. Sim-to-real gap: Training relies entirely on synthetic data, leaving a visual domain gap for real-world deployment; rendering fidelity for transparent glassware and reflective liquids may be insufficient.
  2. Deterministic Action Dictionary: Correction strategies are converted to control primitives via a deterministic Action Dictionary, limiting recovery flexibility and unable to handle open-ended language corrections.
  3. Low absolute success on hard tasks: Pour task reaches only 48% even after +16pp improvement, indicating reliable laboratory recovery remains an open challenge.

Conclusion and Outlook

LabRobFail provides a complete framework for chemical laboratory robotic failure analysis: a physics-based simulation platform (Sim), a 20K+ trajectory dataset (Data), a six-dimensional benchmark (Bench), and a specialized VLM. LabRobFail-VLM substantially outperforms generalist baselines in failure reasoning and improves downstream task success rates by 4-16pp through closed-loop recovery, paving the way for reliable autonomous scientific discovery.

Key Insight: "The irreversibility and safety-criticality of chemical experiments make failure understanding non-optional — LabRobFail transforms failures from unpredictable black-box events into engineerable objects that can be detected, localized, and corrected through three-level controllable injection and six-dimensional structured diagnosis."

Related Papers

Humanoid Seated Locomotion on Passive Mobile Chair

Humanoid Seated Locomotion on Passive Mobile Chair

Research on omnidirectional seated locomotion for humanoids on passive chairs. Policies use proprioception only, outperforming standing in tracking. Analyzes regularization effects on convergence.

人形机器人Humanoid坐姿移动Aug 28, 2026
GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture

GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture

Vision-language-action (VLA) models have become a dominant paradigm for generalist embodied agents, demonstrating strong complex and long-horizon task completion in structured settings. Yet it remains an open question whether current VLA systems can benefit from more effective architectural design, scale to substantially larger and more heterogeneous data regimes, and achieve broader generalization across tasks and embodiments. To this end, we present GigaBrain-0.7, an embodied foundation model with substantially improved generalization across diverse robot embodiments. Specifically, GigaBrain-0.7 unifies understanding, prediction, and action through a three-system architecture, scales pretraining to over 37,000 hours of heterogeneous embodied data, and introduces one-stage alignment training that jointly optimizes vision-language understanding and multi-embodiment action generation. Compared with the preceding GigaBrain-0 series and prior state-of-the-art models including $π_{0.5}$, GigaBrain-0.7 achieves substantial improvements in foundation zero-shot capabilities, language-conditioned instruction following, and post-training task success rates. In particular, on our in-house Maker H01 platform and mainstream robot embodiments, GigaBrain-0.7 demonstrates strong task adaptability and completion ability across both home and industrial scenarios. All training code and pretrained model weights will be released.

VLA具身智能世界模型Aug 16, 2026
ABot-N1: Toward a General Visual Language Navigation Foundation Model

ABot-N1: Toward a General Visual Language Navigation Foundation Model

ABot-N1 is a general visual-language navigation foundation model built on a slow-fast dual-system architecture: a 4B slow VLM reasoner emits explicit chain-of-thought plus dual pixel goals (Target Pixel + Affordance Pixel), while a 2B fast action expert decodes continuous SE(2) waypoints via QFormer action queries. The unified pixel-goal interface covers five tasks — point-goal, object-goal, POI-goal, instruction-following and person-following — in a single 30M-sample multi-task checkpoint, further aligned by GRPO post-training with format/target/safety rewards. Two closed-loop benchmarks are released (ABotN-PointBench and ABotN-POIBench). ABot-N1 sets new SOTA on all five benchmarks, boosting POI entrance arrival to 77.3% (+35.0 pp) and reaching 92.9%/95.4% outdoor/indoor point-goal SR, with full deployment on the TuTu quadruped running on a Jetson AGX Orin.

视觉语言导航VLN导航基础模型Jul 11, 2026
Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence

Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence

Zetta is a closed-loop embodied harness that keeps the base VLA frozen and evolves code-based runtime critics and recovery skills through three timescale-separated loops: action-frequency governance, rollout-batch failure diagnosis, and validation-gated skill updates. With the Z-Infra rollout infrastructure (20.6x throughput), it reaches 90.8% on LIBERO-Pro and 93.6% on RoboCasa, with zero-shot skill transfer and robotic Aha moments.

具身智能Embodied AIVLAAug 17, 2026