PAPER DEEP DIVE
ConsiSpace: Learning Geometric Consistency Matters for Video Spatial Reasoning
Video spatial reasoning is essential for navigation-oriented perception and long-video question answering, where models must infer spatial relations across long horizons under changing viewpoints. However, existing multimodal large language models (MLLMs) remain largely semantic-centric, and often fail to reliably aggregate consistent spatial evidence from redundant video observations, leading to inefficient or unstable reasoning. To address these issues, we propose ConsiSpace, a geometry-consistency-aware framework for geometry-sensitive video spatial reasoning that turns spatial consistency into both an evidence organization principle and an explicit post-SFT learning signal. We build a geometry-consistent memory (GCM) including implicit evidence tokens and explicit geometric cues, and leverage efficient organization strategies to compactly preserve task-related spatial evidence. Furthermore, we utilize unified consistency self-supervised reinforcement learning (UC-SSRL) after supervised fine-tuning to improve cross-view stability, with answer-, metric-, and topology-consistency rewards. Extensive experiments on three spatial-reasoning benchmarks, VSI-Bench, OSI-Bench, and MMSI-Video-Bench, show consistent gains, improving the average score by 12.6 points over the strongest baselines.
ConsiSpace: Learning Geometric Consistency Matters for Video Spatial Reasoning
Paper: ConsiSpace: Learning Geometric Consistency Matters for Video Spatial Reasoning
Authors: Ting Huang, Zhenyu Zhang, Wenyuan Huang, Jian Yang, Hao Tang
Affiliation: Nanjing University, Peking University, Beijing Academy of AI
Link: arXiv:2607.17599 | Project: ConsiSpace
One-Sentence Summary
ConsiSpace is a geometry-consistency-aware framework for video spatial reasoning that organizes implicit evidence tokens and explicit geometric cues through Geometry-Consistent Memory (GCM), and optimizes cross-view stability via Unified Consistency Self-Supervised Reinforcement Learning (UC-SSRL) after SFT, improving average score by 12.6 points on VSI-Bench, OSI-Bench, and MMSI-Video-Bench.
Background and Motivation
Video spatial reasoning is essential for navigation perception and long-video QA — models must infer spatial relations across long horizons under changing viewpoints. Existing MLLMs are semantics-centric, failing to reliably aggregate consistent spatial evidence from redundant video observations. Prior methods (SpaceR, ViLaSR, Cambrian-S) improve through spatial data and training objectives but lack a unified framework treating geometric consistency as both an evidence organization principle and an explicit learning signal.
Figure 1: ConsiSpace framework overview. Organizes visual-spatial tokens and geometric cues via GCM and UC-SSRL.
Method Details
1. Feature Extraction
Built on Qwen3-VL-8B with frozen SigLIP2 as visual encoder and frozen VGGT as geometry encoder. Each frame extracts visual tokens $X_t$, spatial tokens $S_t$, camera pose $T_t$, and depth map $D_t$.
2. Geometry-Consistent Memory (GCM)
GCM has two banks: Implicit Evidence Memory $\mathcal{M}_{\text{imp}}$ stores per-frame evidence tokens; Explicit Spatial Memory $\mathcal{M}_{\text{exp}}$ stores camera poses and depth cues. Evidence entries:
$$ e_t = \text{Pack}(X_t, S_t), \qquad g_t = (T_t, D_t) $$Geometry-gated writing: Compute inter-frame motion and viewpoint changes, write only when new spatial evidence appears:
$$ \Delta p_t = \|p_t - p_{t-1}\|_2, \quad \Delta\theta_t = \angle(v_t, v_{t-1}) $$ $$ \text{write}(t) = \mathbb{I}\left[\Delta p_t > \tau_p \lor \Delta\theta_t > \tau_\theta\right] $$Geometry-consistent fusion: Define geometric neighborhood $\mathcal{N}(t) = \{j: \|p_t - p_j\| < r, \angle(v_t, v_j) < \theta\}$, fuse into semantically most similar neighbor:
$$ j^\star = \arg\max_{j \in \mathcal{N}(t)} \cos(k_t, k_j), \qquad k_t = W_k e_t $$
Figure 2: GCM evidence lifecycle. Geometry-gated writing → consistent fusion → filtered retrieval.
Geometry-filtered retrieval: Coarse-to-fine — retrieve candidate chunks by semantic relevance, then refine frame-level selection with geometry term:
$$ s_f(i) = \alpha \cos(q_e, k_i^f) + \beta \cos(Q_{\text{topo}}, e_i^{\text{dir}}) $$Final context concatenates query, summary, evidence, and geometric cues:
$$ Z = \text{Concat}\left([\text{TOK}(q)],\, z_{\text{topo}},\, z_{\text{metric}},\, \{z_i\}_{i \in F_{\text{topk}}},\, \{g_i\}_{i \in F_{\text{topk}}}\right) $$3. Unified Consistency Self-Supervised RL (UC-SSRL)
Construct two-view observations $o_i$ and $o_j$ for the same question, with three self-supervised rewards:
Answer consistency (KL divergence penalty):
$$ R_{\text{KL}} = -\alpha\, \text{KL}(\pi_i \| \pi_j) $$Metric consistency (distance estimate agreement):
$$ R_{\text{metric}} = -\eta\, |\hat{d}_i - \hat{d}_j| $$Topological consistency (symmetric KL of relational distributions):
$$ R_{\text{topo}} = -\beta\left(\text{KL}(P_i \| P_j) + \text{KL}(P_j \| P_i)\right) $$Unified reward $R = R_{\text{KL}} + R_{\text{metric}} + R_{\text{topo}}$ optimized via policy gradient, updating only LoRA parameters.
flowchart TD
A["Input Video Frames"] --> B["Feature Extraction
SigLIP2(visual)+VGGT(geometry)"]
B --> C["Geometry-Consistent Memory GCM"]
C --> D["Geometry-Gated Writing
Δp>τp ∨ Δθ>τθ"]
D --> E["Geometry-Consistent Fusion
Max semantic similarity in neighborhood"]
E --> F["Geometry-Filtered Retrieval
Coarse-to-fine: semantic+geometry scoring"]
F --> G["Context Concatenation
Query+summary+evidence+geometry"]
G --> H["Video Language Model
Qwen3-VL-8B+LoRA"]
H --> I["SFT Initialization"]
I --> J["UC-SSRL Consistency Optimization
Answer+metric+topology triple rewards"]
J --> K["Cross-view stable spatial reasoning output"]
style C fill:#e1f5fe
style J fill:#fff3e0
style K fill:#e8f5e9
Experimental Results
VSI-Bench Indoor Spatial Reasoning
| Method | Type | Avg. |
|---|---|---|
| GPT-4o | API | 34.0 |
| Gemini-1.5 Pro | API | 45.4 |
| Qwen3-VL-8B | Open-source | ~48 |
| Spatial-MLLM | Specialized | ~52 |
| VLM-3R | Specialized | ~58 |
| ConsiSpace (UC-SSRL) | Ours | ~65 |
MMSI-Video-Bench Multi-Video Spatial Reasoning
| Method | Avg. |
|---|---|
| Gemini 2.5 Flash-Thinking | 36.7 |
| VLM-3R | 42.6 |
| ConsiSpace SFT | 48.4 |
| ConsiSpace UC-SSRL | 57.5 |
ConsiSpace UC-SSRL achieves 57.5 on MMSI-Video-Bench, +14.9 over strongest baseline VLM-3R (42.6). UC-SSRL improves +9.1 over pure SFT, demonstrating consistency RL effectiveness.
Table 1: VSI-Bench detailed results.
Table 3: MMSI-Video-Bench detailed results.
Table 4: End-to-end efficiency and long-video scaling analysis.
Limitations
- Geometry encoder dependency: Relies on frozen VGGT for pose and depth estimation; geometric priors may be unreliable in fast-motion or texture-less scenes.
- Limited outdoor coverage: SFT data is primarily indoor (VSI-590K) with only nuScenes-10K for outdoor; generalization to complex outdoor spatial relations may be insufficient.
- UC-SSRL computational overhead: Two-view sampling and triple KL divergence computation increases training time, though only LoRA parameters are updated.
Conclusion and Outlook
ConsiSpace turns geometric consistency into both an evidence organization principle and an explicit learning signal, achieving geometry-gated writing, consistent fusion, and filtered retrieval through GCM, and optimizing cross-view stability through UC-SSRL. It improves average score by 12.6 points on three spatial reasoning benchmarks, providing a geometry-aware solution for navigation perception and long-video QA.
Key Insight: "Spatial consistency is not just an evidence organization principle but an explicit learning signal — when underlying 3D relations are stable, predictions from different views should agree, and this self-supervised constraint shifts models from semantics-first to geometry-aware."
SOURCE LINKS
