Towards Dual-Brain Minimal Sufficient Representation for Vision-Language Navigation

Figure 1 — Comparison with representative VLN paradigms. (a) VLM-based policies directly map observations and instructions to actions without future modeling. (b) World-model frameworks include future-state prediction but suffer perceptual redundancy. (c) CompactNav introduces minimal sufficient world representation.

1. Background & Motivation

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to follow natural language instructions in unseen 3D environments using egocentric RGB-D observations. Despite progress with multimodal large models and world models, existing methods face three fundamental challenges:

• Semantic Granularity Mismatch: the entropy gap between sparse symbolic instructions and dense perceptual observations causes task-relevant cues to be overwhelmed by visual redundancy — instruction dilution.
• High-entropy redundancy-induced overfitting: fidelity-oriented world models preserve excessive task-irrelevant details (textures, illumination), reducing generalization.
• Inefficient intention-perception coordination: lack of efficient mechanisms to synchronize discrete linguistic intent with continuous perceptual dynamics, causing high computational overhead.

2. Core Method

Figure 2 — CompactNav architecture overview. Language instruction and RGB-D observations go through the Logical Anchor Model to produce instruction-grounded embedding $\mathcal{Z}$, compressed via Minimalist Constraint Alignment, then the Compression World Model performs action-conditioned transition and decodes navigation actions.

2.1 Logical Anchor Model (Focus-Former)

The core challenge is semantic granularity mismatch: instruction $I$ has moderate global entropy, while observation $O$ forms a high-entropy perceptual space. Traditional Q-Former concatenates features $[F^{(\mathcal{I})}, F^{(\mathcal{O})}]$ for global attention, forcing instruction signals to compete with massive observation noise:

$$\sum \text{Attention}(Q_A, F^{(\mathcal{I})}) + \sum \text{Attention}(Q_A, F^{(\mathcal{O})}) = 1$$

Focus-Former reformulates this as a conditional filtering mechanism $P(F^{(\mathcal{I})} \mid Q_B) \cdot P(F^{(\mathcal{O})} \mid F^{(\mathcal{I})}, Q_B)$, using the low-entropy instruction as prior to deconstruct the perceptual bottleneck, in two stages:

• Logic Anchoring: Text Q-Former conditions queries $Q_B$ on instruction features $F^{(\mathcal{I})}$, producing logic-aligned queries $Q_B' = \text{Attn}(Q_B, F^{(\mathcal{I})}, F^{(\mathcal{I})})$.
• Perceptual Selection: Anchored queries $Q_B'$ interact with observation features $F^{(\mathcal{O})}$ — attention becomes targeted selection rather than cross-modal competition.

Distilled representations are fed into MLLM (Qwen2.5-VL-7B): $\mathcal{Z} = \text{MLLM}([\mathbf{H}_B, \mathcal{F}^{(\mathcal{I})}])$.

2.2 Minimalist Constraint Alignment (MCA)

MCA establishes a differentiable low-rank alignment channel via CP Decomposition, approximating the high-order world tensor $\mathcal{Z} \in \mathbb{R}^{N \times M \times D}$ as a sum of $\mathbf{R}$ rank-one tensors from three basis vectors $\mathbf{u}_r, \mathbf{v}_r, \mathbf{w}_r$:

$$\hat{\mathcal{Z}} = \sum_{r=1}^{\mathbf{R}} \mathbf{u}_r \circ \mathbf{v}_r \circ \mathbf{w}_r$$

Reducing representational complexity from $O(NMD)$ to $O(\mathbf{R}(N+M+D))$, creating an explicit information bottleneck.

2.3 Compression World Model (CWM)

CWM performs action-conditioned state transition in a compressed low-rank latent space, maintaining compact predictive belief over future dynamics and decoding the next navigation action. Unlike traditional world models operating on high-dimensional pixel-level reconstructions, CWM works in CP-decomposed factorized latent space.

flowchart LR
    I["Language instruction I"] --> LAM["Logical Anchor Model\n(Focus-Former)"]
    O["RGB-D observation O"] --> LAM
    LAM -->|"Instruction-grounded Z"| MCA["Minimalist Constraint Alignment\n(CP decomposition low-rank)"]
    MCA -->|"Compressed Ẑ"| CWM["Compression World Model\n(action-conditioned transition)"]
    CWM -->|"Navigation action"| A["Action sequence A"]
    CWM -->|"Predictive belief"| CWM
    style LAM fill:#dbeafe,stroke:#2563eb
    style MCA fill:#fef9c3,stroke:#ca8a04
    style CWM fill:#dcfce7,stroke:#16a34a

Figure M1 — CompactNav three-module data flow. Instruction and observation aligned by Logical Anchor Model, compressed via CP decomposition bottleneck, then Compression World Model predicts transitions and decodes navigation actions.

3. Experimental Results

Evaluated on R2R-CE and RxR-CE benchmarks, both instantiated in Habitat on Matterport3D scenes.

3.1 R2R-CE Performance

Table 1 — R2R-CE dataset results. CompactNav outperforms all methods across splits.

CompactNav reaches 66% SR and 54 SPL on Val-Unseen (full), 63% SR and 55 SPL on Test Unseen. NE reduced to 4.36m, TL only 12.56m — avoiding perceptual redundancy of reconstruction-based methods.

3.2 RxR-CE Performance

Table 2 — RxR-CE dataset results. CompactNav achieves SOTA on the more complex long-instruction benchmark.

On the more complex RxR-CE with longer trajectories and instructions, CompactNav achieves SOTA SR 58.96%, SPL 49.76, NDTW 65.67. CWM's recursive memory maintains compact history in low-dimensional manifold, enabling internal path simulation without perceptual detail interference.

3.3 Inference Efficiency

Despite integrating MLLM and world model, CompactNav maintains 236s inference time, comparable to ETPNav-NavMorph (227s). CWM works in factorized latent space, reserving most computational budget for MLLM semantic reasoning.

3.4 Ablation Study

The full model achieves the best accuracy-efficiency trade-off; removing any module degrades performance, confirming module complementarity.

4. Contributions & Significance

• A navigation framework based on the Principle of Minimal Sufficiency, balancing representational fidelity with computational efficiency.
• Focus-Former: reformulates multimodal interaction as instruction-conditioned filtering, resolving semantic granularity mismatch.
• MCA: builds a low-rank information bottleneck via CP decomposition, reducing complexity from $O(NMD)$ to $O(\mathbf{R}(N+M+D))$.
• CWM: performs action-conditioned transition in factorized latent space, enabling MLLM inference and world model coexistence without sacrificing efficiency.

5.

注意力A的计算

$$ \mathbf{H}_{A}=\text{Softmax}\left(\frac{(Q_{A}W_{Q})([F^{(\mathcal{I})},F^{(\mathcal{O})}]W_{K})^{\top}}{\sqrt{d}}\right)([F^{(\mathcal{I})},F^{(\mathcal{O})}]W_{V}) $$

注意力B的计算

$$ \mathbf{H}_{B}=\text{Softmax}\left(\frac{(\text{Attn}(Q_{B},F^{(\mathcal{I})},F^{(\mathcal{I})})W_{Q})(F^{(\mathcal{O})}W_{K})^{\top}}{\sqrt{d}}\right)(F^{(\mathcal{O})}W_{V}) $$

张量分解重建

$$ \hat{\mathcal{Z}}=\sum_{r=1}^{R}\mathbf{u}_{r}\otimes\mathbf{v}_{r}\otimes\mathbf{w}_{r} $$

Limitations

• Relies on MLLM (Qwen2.5-VL-7B) prior knowledge, making the model relatively large.
• CP decomposition rank $\mathbf{R}$ requires hyperparameter tuning; optimal values may differ across tasks.
• In visually monotonous or highly unfamiliar environments, compressed representations may lose critical details.

6. Conclusion

CompactNav, grounded in the Principle of Minimal Sufficiency, synergizes the Logical Anchor Model (Focus-Former instruction-aware filtering), Minimalist Constraint Alignment (CP decomposition low-rank bottleneck), and Compression World Model (low-rank latent transition) to improve SOTA by 2.0%/1.0 and 0.94%/0.78 SR/SPL on R2R-CE and RxR-CE respectively, while maintaining inference efficiency comparable to baselines. Focus-Former uses the low-entropy instruction as prior to deconstruct the perceptual bottleneck; MCA reduces representational complexity to $O(\mathbf{R}(N+M+D))$; CWM efficiently performs transitions in factorized latent space. Navigation doesn't need to remember everything, only the minimally sufficient everything — minimal sufficient world representations provide an effective foundation for robust VLN.