Towards Capability-Aware Traversability Navigation for Unstructured Environments

1. Paper Overview

This paper from the University of São Paulo's Mobile Robotics Group presents CAT (Capability-Aware Traversability), a framework addressing a core contradiction in navigating unstructured environments: the same terrain can be safe for a quadruped robot yet dangerous for a wheeled one. Existing methods typically handle this embodiment difference through late-stage trajectory filtering after perception, rather than encoding platform constraints into the learned representation.

CAT's core innovation is embedding robot physical limits directly into the spatial feature space, modulating semantic terrain maps with robot-specific traversability vectors through Spatially-Adaptive Denormalization (SPADE) blocks. The framework supports four robot profiles: wheeled (TerraSentia), legged (ANYmal), differential, and ATV (TartanDrive). On human-annotated and trajectory-aligned datasets, CAT leads all ranking-based metrics — improving AUROC by 11.0% on physically executed trajectories and AUPRC by 15.8% on human traces over the strongest baseline.

2. Problem Background and Motivation

Traversability estimation projects environmental features into navigation costs to determine the feasibility of traversing specific terrain for a given robot morphology. However, merging interaction experiences across multiple platforms introduces conflicting labels: a quadruped's trajectory navigating stairs is fundamentally unsuitable for a wheeled robot. Such embodiment-specific contradictions prevent creating a shared traversability representation and severely limit policy transferability across robotics platforms.

Existing methods suffer from two main shortcomings:

• Methods relying solely on prior robot experience degrade significantly outside specific training domains.
• VLM zero-shot approaches, while possessing strong semantic reasoning, apply only to robot-agnostic scene context — embodiment-specific feasibility never enters the learned representation. Current pipelines integrate a secondary filter after perception to discard infeasible paths, rather than encoding embodiment in the representation itself.

The paper's key insight is that although robotics platforms have different embodiment profiles, the underlying relationship between semantic terrain classes and physical interaction admits a shared structure. Perception models can internalize physical limits by actively modulating visual features based on robot-specific profiles.

3. CAT Framework in Detail

3.1 Overall Architecture

The CAT framework contains three main components:

1. Embedding Generation: DINOv3 extracts dense visual features
2. Semantic Terrain Modulation: CLIPSeg generates semantic terrain maps, modulated by robot traversability vectors through SPADE blocks
3. Traversability Prediction: cosine similarity computes cost maps in [0,1] range

3.2 Semantic Terrain Modulation

CAT uses CLIPSeg to generate semantic terrain segmentation maps, identifying 10 terrain classes. Then through SPADE (Spatially-Adaptive Denormalization) blocks, robot-specific traversability vectors are fused with semantic terrain maps:

graph LR
    A["RGB-D Input"] --> B["DINOv3
Dense feature extraction"]
    A --> C["CLIPSeg
Semantic terrain segmentation"]
    D["Robot traversability vector z_r"] --> E["SPADE modulation block"]
    C --> E
    B --> E
    E --> F["Cosine similarity"]
    F --> G["Traversability cost map
[0,1]"]

The SPADE block performs spatially-adaptive normalization, where $\mu$ and $\sigma$ are conditioned on the semantic terrain map and traversability vector:

$$\hat{y} = \frac{y - \mu}{\sigma}, \quad \hat{y}' = \gamma \odot \hat{y} + \beta$$

where $\gamma$ and $\beta$ are learned scaling and shift parameters derived from the traversability vector $\mathbf{z}_r$, and $\mu$ and $\sigma$ are spatial conditional statistics. This enables the same terrain to produce different traversability predictions under different robot profiles.

The traversability cost is computed via cosine similarity between the decoded feature $\mathbf{u}_j$ and the robot prototype $\mathbf{z}_r$, with $L_2$ normalization:

$$T(\mathbf{u}_j, \mathbf{z}_r) = \frac{\mathbf{u}_j \cdot \mathbf{z}_r}{\|\mathbf{u}_j\| \cdot \|\mathbf{z}_r\|} \in [-1, 1]$$

Scores are normalized to $[0, 1]$ for visualization, with blue denoting traversable regions and red denoting obstacles.

3.3 Trajectory-Grounded Label Generation

CAT grounds dense supervision masks in physical trajectories through an interactive annotation pipeline. The key is positive sample refinement: when a pixel's modulated suitability falls below a fixed threshold, it is demoted to a negative. With $T_{\text{thresh}} = 0.4$, the refinement condition is:

$$\text{label}(p) = \begin{cases} \text{positive} & \text{if } T(p) \geq T_{\text{thresh}} \\ \text{negative} & \text{if } T(p) < T_{\text{thresh}} \end{cases}$$

When a pixel's modulated suitability falls below the fixed threshold, it is demoted to a negative — regions marked traversable by segmentation but deemed impassable by the robot's traversability vector do not serve as positives.

3.4 Training Objective

CAT uses a joint InfoNCE loss function:

$$\mathcal{L} = (1 - \omega)\,\mathcal{L}_{\mathrm{traj}} + \omega\,\mathcal{L}_{\mathrm{mask}}$$

where $\mathcal{L}_{\mathrm{traj}}$ is the trajectory term (positives sampled exclusively from robot path pixels) and $\mathcal{L}_{\mathrm{mask}}$ is the mask term (sampled from dense trajectory-grounded mask pixels), with $\omega = 0.05$. The InfoNCE objective pulls positive spatial features $\mathbf{u}_j$ closer to the prototype $\mathbf{z}_r$ while pushing negative features away. Per batch, 256 positives and 1024 negatives are sampled (trajectory mask), plus 512 positives and 1024 negatives (generated mask).

The InfoNCE contrastive prototype loss takes the form:

$$\mathcal{L}_{\mathrm{InfoNCE}} = -\log \frac{\exp(\text{sim}(\mathbf{u}_j, \mathbf{z}_r) / \tau)}{\sum_{k} \exp(\text{sim}(\mathbf{u}_k, \mathbf{z}_r) / \tau)}$$

where $\tau = 0.07$ is the temperature parameter and $\text{sim}(\cdot, \cdot)$ is cosine similarity.

4. Experimental Results

4.1 Quantitative Comparison

Key findings:

• On the held-out trajectory split, CAT leads all metrics, improving AUROC by 11.0% over W-RIZZ (0.945 vs 0.851).
• On NaviTrace, CAT achieves the highest AUROC and AUPRC, with AUPRC improving 15.8% over WayFAST (RGB-D) (0.169 vs 0.146).
• The Exterior Activation rate (Ext. Act.@90R) measures the fraction of exterior pixels activated at the threshold retaining 90% positive recall; CAT achieves only 0.143 on the held-out set:

  $$\text{Ext.Act.}_{90} = \frac{|\{p \in \Omega_{\text{ext}} : T(p) \geq \theta_{90}\}|}{|\Omega_{\text{ext}}|}$$

• WayFAST (RGB-D) assigns higher average scores within trace masks, but its lower AUPRC and higher exterior activation indicate overly broad responses.

4.2 Capability-Conditioned Modulation

The paper demonstrates traversability maps generated from the same RGB-D observation under two robot profiles:

• Wheeled profile: assigns high traversability primarily to flat terrain, avoiding stairs and rough surfaces
• Legged profile: expands the traversable region to include stairs and uneven terrain

Embodiment AUROC measures whether each profile-conditioned map ranks the region exclusive to the queried profile above the region exclusive to the other. Results show CAT genuinely adjusts predictions based on robot capabilities, rather than producing generic path predictions.

4.3 Real-World Deployment

CAT is validated on two robot platforms: a legged quadruped and a wheeled skid-steer. It runs on embedded hardware at 4.8 Hz, achieving embodiment-aware obstacle avoidance. The model has only 8.74M trainable parameters, with DINOv3 and CLIPSeg remaining frozen. Training takes 35 epochs on a single NVIDIA L40S GPU (~8 hours).

5. Limitations and Shortcomings

• Evaluation metric limitations: Path-vs-Rest metrics are path-alignment measures rather than safety classification, since pixels outside masks remain unobserved under positive-unlabeled supervision and may still be traversable.
• Training data bias: The NaviTrace human-annotated dataset may contain human bias, as human-judged safe regions may not fully match actual robot capabilities.
• Limited robot profiles: Only four profiles are trained (wheeled, legged, differential, ATV), not covering aerial, tracked, or other morphologies.
• Limited terrain classes: Only 10 terrain classes, which may be insufficient for extremely complex environments.
• Embedded hardware framerate: 4.8 Hz inference rate may not be sufficiently real-time for high-speed motion scenarios.

6. Summary

CAT's core contribution lies in demonstrating that traversability need not be robot-agnostic — by encoding physical limits directly into spatial feature representations rather than relying on post-hoc trajectory filtering. The SPADE modulation mechanism enables a single model to produce different traversability predictions based on robot profiles, exhibiting capability-aware behavior in both quantitative metrics and qualitative visualization. Jointly training four robot profiles, CAT achieves 0.945 AUROC on the held-out trajectory split, an 11.0% improvement over the strongest baseline.

More importantly, ablation experiments demonstrate that spatial conditioning and per-robot prototypes genuinely produce capability sensitivity beyond generic path prediction. Deployment on real quadruped and wheeled robots validates the framework's practicality. Future work can extend to more robot morphologies and terrain classes, and explore end-to-end optimization with downstream path planning.