Token-Wise Latent Streaming from Slow Reasoners to Fast Planners for Dynamic Vision Language Navigation

Paper Metadata

Title: Token-Wise Latent Streaming from Slow Reasoners to Fast Planners for Dynamic Vision Language Navigation (SPARK-VLN)

Authors: Tianshuai Hu, Yangyi Zhong, Zeying Gong, Lingdong Kong, Xiaodong Mei, Guoyang Zhao, Xiaolu Liu, Song Wang, Rong Li, Junwei Liang (corresponding author). Affiliations: The Hong Kong University of Science and Technology (Guangzhou), HKUST, and Zhejiang University.

Links: arXiv:2607.16806 (https://arxiv.org/abs/2607.16806 , v1, July 18, 2026); project page https://hutslib.github.io/SPARK-VLN/

Code and data status: at the time of this review the Code button on the project page is still a placeholder pointing at #, the authors' GitHub account hosts no SPARK-VLN repository, and the paper lists no download for the code or the benchmark. The implementation details given in the text (hidden layers 16-23, N=8 slots, 5-step Euler integration, single-RTX-4090 latency) are specific enough to reimplement, but neither the official code nor the dynamic social VLN suite has been released.

Fig. 1: the temporal structure of three execution paradigms. (a) Single-system reason-then-act: the VLM must finish inference before emitting an action, so the agent idles while the environment advances. (b) Dual-system with blocking update: the expert planner acts continuously but receives guidance (yellow arrows) only after the VLM finishes, so the conditioning signal is already stale. (c) SPARK-VLN token streaming: hidden states flow to the planner throughout inference (blue arrows), giving progressively refined guidance.

One-Sentence Summary

SPARK-VLN replaces the handover in dual-system navigation, from "tell the fast planner once the slow reasoner has finished speaking" to "tell it while the reasoner is still speaking": a frozen VILA-8B hands the hidden states of layers 16-23 to a Perceiver-style cross-attention bridge at every generated token, which compresses them into 8 fixed latent slots that drive a rectified-flow-matching planner in real time, cutting per-update latency from 0.788 s to 0.185 s and reaching 41.60% / 34.80% success rate in the idealized / realistic stages of a dynamic social VLN benchmark.

Background and Motivation

Put instruction following inside an indoor space where people walk, and the first obstacle is not semantic but temporal. Reasoning in a multimodal large model is inherently slow: a 7B-8B backbone has to encode vision for an instruction and then autoregressively decode a string of answer tokens, which costs hundreds of milliseconds even on a consumer GPU. Pedestrians do not wait. The paper names the resulting perception-action mismatch observation staleness: an action is computed from an observation the world has already outrun, so a maneuver that was socially appropriate when inference began can turn into a proxemic violation, or an outright collision, by the time it executes. In dynamic, human-centric settings this gap is safety-critical rather than merely an efficiency number.

None of the three existing routes addresses it head on. Single-system reason-then-act stacks (NaVid, Uni-NaVid, NaVILA) ignore latency altogether and stall between actions, handing the entire window of environment evolution to the pedestrians. Real-time chunking and continuous-inference pipelines overlap inference with execution: motion continuity is preserved but semantic freshness is not, since every chunk stays anchored to the frame captured at the start of a long inference cycle. Dual-system designs (DualVLN, StreamVLN and kin) decouple a slow reasoner from a fast policy running at different frequencies, recovering reactivity, yet the two still communicate exactly once, when the completed reasoning chain arrives. Guidance is withheld until reasoning finishes, and by then the scene it was reasoned about has changed.

The authors' key observation is that autoregressive inference is incremental. Long before a VLM finishes its answer, the hidden states of its intermediate tokens already encode action-relevant cues such as directional intent and pedestrian context. The bottleneck is therefore not the reasoning process but the delivery of its result: usable knowledge emerges token by token, while delivery happens once. Make delivery token-wise as well, and the reasoning value of the slow system is redeemed continuously inside the inference window instead of being discounted at the moment it lands.

The second motivation concerns evaluation, and it is the more insidious one. Mainstream VLN benchmarks (R2R, RxR, VLN-CE) are static and synchronous: the simulator freezes while the model thinks and advances only after an action is emitted. Inference latency never enters the causal chain, so observation staleness is neither penalized nor measured. Methods that would fail in a live dynamic scene are indistinguishable from methods that would not. This is not an artifact of one baseline's implementation but a protocol-level blind spot: blocking simulation switches off precisely the dimension that a dual-system design most needs to be judged on.

The paper accordingly delivers two things. On the method side, SPARK-VLN (Streams Progressively Aggregated latent Reasoning Knowledge) rewrites the whole extract-project-infuse pipeline into a token-level streaming process realized by three coupling modules. On the evaluation side, it introduces an asynchronous, human-centric benchmark for dynamic social VLN in which pedestrians keep moving while the model infers, goal-driven pedestrians avoid one another with ORCA, and the metrics cover task completion, social compliance, and explicit staleness statistics. The two are mutually dependent: without the second, the gains of the first would be almost invisible under existing protocols.

Fig. 2: framework overview. Top: the VLM backbone processes egocentric RGB and instruction tokens while the Token-Wise Hidden Streamer extracts hidden states from designated layers during autoregressive generation, producing an evolving stream. Middle: the Sequence-to-Slot Latent Bridge augments the stream with layer and time positional encodings, projects it into the planner's embedding space, and compresses it into fixed-size latent slots by cross-attention with learnable queries. Bottom: the expert planner fuses observation and slot embeddings, generates candidate trajectories with rectified flow matching, and selects the action through a safety critic. As the VLM keeps generating, the slots are recomputed and the planner re-plans.

Preliminaries

Dual-system VLN, formalized. Given a natural-language instruction $\mathcal{I}$, the agent navigates to a goal region $\mathcal{G}(\mathcal{I})$. At each discrete step $t$ it receives an egocentric observation and pose

$$x_{t}=(o_{t},\,p_{t}),\qquad o_{t}=(I_{t}^{\mathrm{rgb}},\,I_{t}^{\mathrm{depth}})$$

A policy $\pi$ maps the instruction and observation history to a navigation command $\mathbf{u}_{t}=\pi(\mathcal{I},\,x_{\leq t})$, optimized against the standard VLN objective $\pi^{\star}=\arg\max_{\pi}\mathbb{E}\bigl[M(\Gamma_{\pi},\,\mathcal{G}(\mathcal{I}))\bigr]$, where $M$ aggregates Success Rate, SPL and related metrics. A dual-system policy splits into a slow reasoner $f_{\mathrm{vlm}}$ and a fast expert planner $f_{\mathrm{plan}}$:

$$f_{\mathrm{vlm}}\!\bigl(\mathcal{I},\,x_{t'}\bigr)\mapsto\mathbf{z}_{t'},\qquad \mathbf{c}_{t}=\phi(\mathbf{z}_{t'}),\qquad f_{\mathrm{plan}}\!\bigl(x_{t},\,\mathbf{c}_{t}\bigr)\mapsto\mathbf{u}_{t}$$

Here $t'$ is the instant the VLM starts inference, $\mathbf{z}_{t'}$ is the semantic reasoning state extracted from it, and $\phi$ projects that state into a planner-compatible conditioning signal $\mathbf{c}_{t}$. The weakness of conventional designs is visible in this notation: at time $t$ the planner receives a $\mathbf{c}_{t}$ derived from the older observation $x_{t'}$, with a lag of $t-t'$, and it receives it only after full inference. SPARK-VLN rewrites $\phi$ into a streaming version $\phi_{\mathrm{stream}}$ whose conditioning signal is taken from the tokens generated so far:

$$\mathbf{c}_{t}=\phi_{\mathrm{stream}}\bigl(\mathbf{H}_{t'}^{(1:k_{t})}\bigr),\qquad \mathbf{u}_{t}=f_{\mathrm{plan}}(x_{t},\,\mathbf{c}_{t})$$

where $k_{t}\leq K$ is the latest token emitted by time $t$ out of $K$ total tokens. The planner is therefore steered by progressively refined guidance instead of staying blind for the whole inference period.

Rectified flow matching. The fast planner needs a continuous trajectory generator that samples in few steps while retaining multimodality. The paper follows flow-based VLA policies such as $\pi_{0}$ rather than a diffusion formulation: a linear interpolant is defined between noise and expert trajectory, the network regresses the constant velocity along that path, and a handful of Euler steps suffices at deployment. Straight transport paths are what buy the few-step integration, and few-step integration is what makes real-time operation possible. That matters here specifically because the condition changes on every emitted token, so every change triggers a fresh plan and sampling must be cheap.

Method in Detail

Fig. 3: the full data flow. The slow VLM reasoner takes the instruction plus the 8 most recent egocentric RGB frames. While it autoregressively generates an answer (for example "The next action is move forward 75 cm."), the Token-Wise Hidden Streamer extracts intermediate-layer hidden states and the Sequence-to-Slot Latent Bridge compresses the growing stream into N=8 fixed latent slots that are streamed to the planner. The fast expert planner conditions on the current egocentric RGB-D observation fused with those slots, generates multiple candidate trajectories via rectified flow matching, and a safety critic supervised by the Euclidean Signed Distance Field (ESDF) scores the candidates and picks the executed one.

Overall structure. SPARK-VLN is a dual-system framework in which a slow VLM reasoner and a fast expert planner run concurrently, joined by a token-wise streaming knowledge-transfer pipeline. At each timestep the robot receives an egocentric RGB-D observation and an instruction. The reasoner (VILA-8B, frozen) processes the visual input with the instruction and starts autoregressive generation; the Streamer extracts intermediate hidden states $\mathbf{h}_{\mathcal{L}}^{(k)}$ during generation, yielding a growing prefix; the Latent Bridge projects that variable-length, high-dimensional prefix into a compact set of latent slots living in the planner's conditioning space; the Evolving Latent Conditioner infuses the slots into the rectified-flow-matching planner, which emits a continuous trajectory. VLM keeps generating, the prefix grows, the slots are recomputed, the planner re-plans. This loop repeats many times within a single inference pass.

Module 1: Token-Wise Hidden Streamer. Given the multimodal input $\mathbf{x}_{t'}$, the VLM backbone autoregressively generates $K$ tokens. At decoding step $k\in\{1,\dots,K\}$ the hidden state of the $l$-th transformer layer is a function of the historical context:

$$\mathbf{h}_{l}^{(k)}=\mathbf{F}\bigl(\mathbf{x}_{t'}\mid l,k,\theta\bigr)\in\mathbb{R}^{D}$$

$D$ is the VLM hidden dimension, $\theta$ the frozen VLM parameters, and $\mathbf{F}(\cdot\mid l,k,\theta)$ extracts the feature at layer $l$, decoding step $k$. Standard frameworks block downstream execution until all $K$ tokens exist; the Streamer instead pulls these internal representations out on the fly. Concretely, it designates a subset of decoder layers $\mathcal{L}=\{l_{1},\dots,l_{L}\}$ (layers 16-23 in the implementation, so $L=8$) and stacks the hidden states of every layer in $\mathcal{L}$ for the token generated at step $k$ into a per-token multi-layer representation:

$$\mathbf{h}_{\mathcal{L}}^{(k)}=\bigl[\mathbf{h}_{l_{1}}^{(k)};\,\mathbf{h}_{l_{2}}^{(k)};\,\dots;\,\mathbf{h}_{l_{L}}^{(k)}\bigr]\in\mathbb{R}^{L\times D}$$

By the time decoding has reached step $t$, $k_{t}$ tokens have been emitted, and the cumulative stream available downstream is

$$\mathbf{H}_{t'}^{(k_{t})}=\bigl[\mathbf{h}_{\mathcal{L}}^{(1)},\,\mathbf{h}_{\mathcal{L}}^{(2)},\,\dots,\,\mathbf{h}_{\mathcal{L}}^{(k_{t})}\bigr]\in\mathbb{R}^{L\times k_{t}\times D}$$

Two of its dimensions move: $k_{t}$ grows with generation, and $D$ is the high-dimensional hidden space of an 8B model. Choosing intermediate rather than final layers is deliberate. The last layer has been pushed by the training objective toward the vocabulary distribution, whereas intermediate layers retain more representation entangled with visual scene content and directional intent. The paper does not ablate this layer range, which is picked up again in the limitations.

Module 2: Sequence-to-Slot Latent Bridge. The planner demands a fixed-size, lower-dimensional input, while $\mathbf{H}_{t'}^{(k_{t})}$ is variable-length and high-dimensional; this module is the cross-modal translator. Raw hidden states carry no explicit marker of which layer or which generation step they came from, so each element is first augmented with factorized spatio-temporal positional encodings:

$$\mathbf{m}_{l,k}=\mathbf{h}_{l}^{(k)}+\mathbf{e}_{\mathrm{layer}}(l)+\mathbf{e}_{\mathrm{time}}(k)$$

with learnable $\mathbf{e}_{\mathrm{layer}}(l),\mathbf{e}_{\mathrm{time}}(k)\in\mathbb{R}^{D}$ for the layer index and the decoding step. The enriched tokens are collected and flattened into a unified memory tensor $\mathbf{M}_{t'}^{(k_{t})}\in\mathbb{R}^{(L\cdot k_{t})\times D}$. A learnable projection $\mathbf{W}_{p}\in\mathbb{R}^{D\times d}$ then maps that memory from the reasoning space into the planning space:

$$\widetilde{\mathbf{M}}_{t'}^{(k_{t})}=\mathbf{M}_{t'}^{(k_{t})}\mathbf{W}_{p}\in\mathbb{R}^{(L\cdot k_{t})\times d}$$

Compression to fixed capacity uses a Perceiver-style cross-attention bottleneck: $N$ learnable query vectors $\mathbf{Q}=[\mathbf{q}_{1},\dots,\mathbf{q}_{N}]^{\top}\in\mathbb{R}^{N\times d}$ define the latent slots, and the $i$-th slot is updated as

$$\mathbf{z}_{i}=\mathbf{q}_{i}+\operatorname{CrossAttn}\bigl(\mathbf{q}_{i},\,\widetilde{\mathbf{M}}_{t'}^{(k_{t})}\bigr)$$

followed by a standard feed-forward network. Stacking the slots gives the fixed-dimensional conditioning matrix

$$\mathbf{R}_{t}=\bigl[\mathbf{z}_{1},\,\mathbf{z}_{2},\,\dots,\,\mathbf{z}_{N}\bigr]^{\top}\in\mathbb{R}^{N\times d}$$

with $N=8$ in practice. The point of the design is to decouple the planner's input size from the VLM's output length: whether 3 or 30 tokens have been generated, the planner always sees an $8\times d$ matrix. Compared with global average pooling into a single vector, cross-attention preserves token-level granularity and lets each slot harvest the relevant fragment of spatio-temporal memory on its own. The ablation puts that choice at 9.60 points of success rate.

Module 3: Evolving Latent Conditioner and expert planner. Reactive navigation is modeled as sampling a future trajectory $\tau\in\mathbb{R}^{H\times3}$ of $H$ waypoints from a conditional generator:

$$\tau_{1}=G_{\psi}\bigl(\tau_{0},\,\mathbf{c}_{t}\bigr),\qquad \tau_{0}\sim\mathcal{N}(0,\mathbf{I})$$

$\tau_{1}$ is the generated trajectory and the noise realization $\tau_{0}$ preserves the multimodality of viable paths in dense social space. The planner is driven by a unified conditioning matrix $\mathbf{c}_{t}=[\mathbf{z}_{\mathrm{obs}};\,\mathbf{z}_{\mathrm{g}}^{(m)}]$ that pairs egocentric spatial tokens $\mathbf{z}_{\mathrm{obs}}$ with a goal token $\mathbf{z}_{\mathrm{g}}^{(m)}$ that switches with the task modality $m$:

$$\mathbf{z}_{\mathrm{g}}^{(m)}=\begin{cases}\varnothing,&m=\text{no-goal}\\[2pt] \mathbf{W}_{g}\,g,&m=\text{point-goal}\\[2pt] \mathbf{R}_{t},&m=\text{language-goal}\end{cases}$$

This unified form is what lets the same planner learn foundational navigation and obstacle avoidance from abundant point-goal data and then transfer to VLN: for the language-goal branch the streamed slots $\mathbf{R}_{t}$ simply occupy the slot where a goal coordinate would go, and because $\mathbf{R}_{t}$ is recomputed as generation proceeds, the conditioning context evolves smoothly inside a single VLM reasoning pass. At VLN evaluation time the planner is given no goal coordinate at all (set to zero) and is driven purely by the instruction through the streamed slots.

The generator $G_{\psi}$ is parameterized by conditional rectified flow matching. Over flow time $s\in[0,1]$ a linear interpolant is defined between noise and expert trajectory:

$$\tau_{s}=(1-s)\,\tau_{0}+s\,\tau_{1}$$

whose theoretical target velocity $\tau_{1}-\tau_{0}$ is constant across $s$. A transformer-based vector field network $v_{\psi}$ regresses that velocity by minimizing the empirical rectifying loss

$$\mathcal{L}_{\mathrm{RF}}=\mathbb{E}_{s\sim\mathcal{U}[0,1],\,\tau_{0},\,\tau_{1}}\Bigl[\bigl\|v_{\psi}(s,\,\tau_{s},\,\mathbf{c}_{t})-(\tau_{1}-\tau_{0})\bigr\|^{2}\Bigr]$$

The straight transport path admits accurate few-step integration. At deployment each candidate is decoded by $J$-step Euler integration ($J=5$) using the latest $\mathbf{c}_{t}$:

$$\tau^{(j+1)}=\tau^{(j)}+\tfrac{1}{J}\,v_{\psi}\Bigl(\tfrac{j}{J},\,\tau^{(j)},\,\mathbf{c}_{t}\Bigr)$$

Sampling several noise realizations yields $N_{c}$ candidate trajectories $\{\tau^{[n]}\}_{n=1}^{N_{c}}$, from which a goal-agnostic safety critic selects the one that executes:

$$\tau^{*}=\arg\max_{\tau^{[n]}}C_{\omega}\bigl(\tau^{[n]},\,\mathbf{z}_{\mathrm{obs}}\bigr)$$

$C_{\omega}$ is supervised by the Euclidean Signed Distance Field and scores collision-freeness and kinematic smoothness only, never goal attainment. This is a second separation of duties: reactive safety filtering is isolated from high-level intent generation. The critic does not need to understand language, and the language guidance does not need to worry about collisions; the two meet only at candidate selection.

flowchart TD
  A["Language instruction + 8 recent egocentric RGB frames"] --> B["Slow reasoner VILA-8B frozen"]
  B --> C["Token-Wise Hidden Streamer
layers 16-23 per token
cumulative stream H of shape L x k_t x D"]
  C --> D["Add layer and time positional encodings"]
  D --> E["Projection W_p from D dims to d dims"]
  E --> F["N=8 learnable queries cross-attention
fixed latent slots R_t of shape 8 x d"]
  G["Current egocentric RGB-D"] --> H["Observation tokens z_obs"]
  F --> I["Evolving Latent Conditioner
goal token picks none or point goal or R_t"]
  H --> I
  I --> J["Rectified flow matching planner
5-step Euler integration over N_c candidates"]
  J --> K["Goal-agnostic safety critic
ESDF supervised collision free and smooth"]
  K --> L["Executed trajectory tau star"]
  L --> G
  C -. "VLM still decoding so the prefix grows" .-> F
  F -. "slots recomputed triggering a re-plan" .-> J

Fig. 4: the streaming loop of SPARK-VLN. The dashed edges are what separates this design from a blocking dual system: every additional token the VLM emits lengthens the hidden prefix, recomputes the slots, and triggers a re-plan, without waiting for generation to end.

Experimental Results

Fig. 5: benchmark overview. (a) Diverse indoor scenes (bedroom, office, hall, corridor) and four human-encounter patterns (frontal approach, intersection, following, corner), with the robot path in green and the pedestrian path in red; the same scenes support both PointNav and VLN under the idealized and realistic evaluation modes. (b) Instruction word cloud colored by semantic role (action, direction, place, object). (c) Spatial statistics: median start-to-goal geodesic distance 8.2 m and median per-pedestrian patrol path length 17.4 m, showing that pedestrian trajectories are long enough to intersect the agent's route repeatedly.

Benchmark design. The two stages are the core of the suite. In the Idealized Dynamic Environment, pedestrians move under ORCA but the simulator pauses during each inference step, so the agent always acts on a fresh observation; this is the latency-free setting. In the Realistic Dynamic Environment, the simulator runs continuously, pedestrians advance while the model computes, and inference delay feeds directly into the outcome. Contrasting the two quantifies each method's robustness to observation staleness. Pedestrian density is scaled to scene area to avoid artificial overcrowding, and the instruction vocabulary covers action, direction, place and object roles. Metrics fall in three groups: task completion (SR, NE, SPL), social interaction (PSC for personal-space compliance, Col for human collision rate, AvgD for average distance to the nearest pedestrian), and inference-speed statistics.

Main VLN results. In the idealized stage SPARK-VLN attains the best SR of 41.60% and SPL of 34.67%, together with the best PSC of 96.2% and the lowest NE of 5.95 m: 4.4 SR points above the strongest single-system baseline NaVILA and 15.4 points above the dual-system baseline DualVLN. Moving to the realistic stage, every method degrades once observations go stale, confirming staleness as a real threat; the heavier reasoners suffer most, with video-based VLMs such as NaVid losing close to a third of their SR. SPARK-VLN sheds only about one sixth and still leads with 34.80% SR and 28.68% SPL, while topping every social metric: PSC 93.9%, collision rate 29.3% (lowest), AvgD 5.13 m (highest).

Method	Type / backbone	Stage	SR ↑	NE ↓	SPL ↑	PSC ↑	Col ↓	AvgD ↑
Seq2Seq	single-system / no MLLM	idealized	17.60	8.32	16.61	94.4	17.3	4.28
CMA	single-system / no MLLM	idealized	26.00	7.27	24.25	94.2	13.8	4.26
NaVid	single-system / LLaMA-VID 7B	idealized	36.40	5.96	31.19	96.1	11.8	4.25
Uni-NaVid	single-system / LLaMA-VID 7B	idealized	36.00	5.98	31.97	94.5	21.6	4.15
NaVILA	single-system / VILA 8B	idealized	37.20	5.99	34.47	93.6	25.4	4.31
DualVLN	dual-system / Qwen-VL-2.5 7B	idealized	26.20	5.96	24.33	93.3	42.0	4.43
SPARK-VLN	streaming dual-system / VILA 8B	idealized	41.60	5.95	34.67	96.2	24.8	4.45
NaVid	single-system / LLaMA-VID 7B	realistic	25.40	6.24	22.35	93.2	32.1	4.24
Uni-NaVid	single-system / LLaMA-VID 7B	realistic	30.20	6.23	26.93	92.9	34.6	4.10
NaVILA	single-system / VILA 8B	realistic	29.00	6.22	26.63	92.5	39.3	4.32
DualVLN	dual-system / Qwen-VL-2.5 7B	realistic	22.90	6.23	21.23	92.6	46.6	4.66
SPARK-VLN	streaming dual-system / VILA 8B	realistic	34.80	6.21	28.68	93.9	29.3	5.13

Table 1: VLN evaluation on the dynamic social benchmark (paper TABLE I). Idealized means pedestrians move but simulation pauses during inference; realistic means the simulator runs continuously. Bold marks the best value within each stage. Note that in the idealized stage SPARK-VLN's collision rate of 24.8 is well above NaVid's 11.8: without latency pressure it is not more conservative than a light single-system model, and its advantage comes mainly from the realistic stage.

Point-goal social navigation. The authors also evaluate the PointNav-pretrained fast planner alone (Ours-S1) to test how solid the substrate under the streamed guidance really is. In the idealized stage Ours-S1 reaches the best learning-based SR of 70.7% and SPL of 70.1%, ahead of NavDP by 9.6 and 9.9 points, with the best social compliance as well: the collision rate drops to 5.9%, nearly half of NavDP's 9.8%. In the realistic stage all methods degrade, yet Ours-S1 keeps a clear lead with 61.2% SR and 60.8% SPL against NavDP's 53.2% and 52.6%. A telling comparison: the oracle PathFollower, which has privileged access to the environment mesh and a mesh-derived reference path, drops from 60.0% to 56.7% SR, while the learned planner beats it in both stages.

Method	Category	Stage	SR ↑	SPL ↑	PSC ↑	Col ↓	AvgD ↑
PathFollower	rule-based / oracle mesh path	idealized	60.0	55.9	93.6	33.9	4.05
ORCA	rule-based / oracle mesh path	idealized	33.2	32.9	96.2	16.4	4.08
iPlanner	learning-based	idealized	49.1	49.1	96.7	10.7	4.53
NavDP	learning-based / diffusion	idealized	61.1	60.2	97.1	9.8	4.54
Ours-S1	learning-based / flow matching	idealized	70.7	70.1	97.2	5.9	4.55
PathFollower	rule-based / oracle	realistic	56.7	52.8	93.0	37.6	4.05
ORCA	rule-based / oracle	realistic	30.0	29.7	95.4	21.5	4.11
iPlanner	learning-based	realistic	43.8	43.8	95.6	20.9	4.64
NavDP	learning-based / diffusion	realistic	53.2	52.6	95.5	21.9	4.60
Ours-S1	learning-based / flow matching	realistic	61.2	60.8	95.8	17.5	4.63

Table 2: point-goal social navigation (paper TABLE II). Oracle methods use a privileged environment mesh and a mesh-derived reference path, so they are not direct competitors.

Ablation 1: knowledge-transfer strategy. Wait-then-Act (W-t-A) blocks the planner until the VLM produces its full output and then infuses a single conditioning signal once per slow cycle, exactly the conventional dual-system arrangement. Stream updates the conditioning continuously through the Streamer and the Latent Bridge. Both share the same slow reasoner and the same fast expert planner, so the transfer mechanism is the only difference. In the realistic stage, where latency actually matters, Stream lifts SR from 24.80% to 34.80% (+10.00), SPL from 21.64% to 28.68% (+7.04), and lowers NE from 7.45 m to 6.21 m; social metrics move the same way, with PSC up to 93.9%, collision rate down to 29.3%, and AvgD up to 5.13 m. This is a clean single-variable control: the gain can only come from when guidance is delivered, not from model capacity or training data.

Ablation 2: bridge aggregation. Mean-Pool collapses all streamed tokens into a single vector by global average pooling, discarding token-level granularity. Cross-Attn lets $N=8$ learnable queries attend to the projected stream and route the most informative features into fixed-size slots. Again in the realistic stage, Cross-Attn improves SR from 25.20% to 34.80% (+9.60) and SPL from 21.17% to 28.68% (+7.51), lowers NE from 7.36 m to 6.21 m, and improves every social metric, with the collision rate falling from 36.5% to 29.3% (-7.2 points, the largest single improvement across both ablations). That collision rate is this sensitive to the aggregation rule suggests average pooling destroys exactly the localized information about which pedestrian is coming from which direction.

Ablation	Variant	SR ↑	NE ↓	SPL ↑	PSC ↑	Col ↓	AvgD ↑
knowledge transfer	W-t-A (blocking)	24.80	7.45	21.64	90.9	29.7	4.97
	Stream (ours)	34.80	6.21	28.68	93.9	29.3	5.13
	Δ	+10.00	-1.24	+7.04	+3.0	-0.4	+0.16
bridge aggregation	Mean-Pool	25.20	7.36	21.17	92.7	36.5	5.05
	Cross-Attn (ours)	34.80	6.21	28.68	93.9	29.3	5.13
	Δ	+9.60	-1.15	+7.51	+1.2	-7.2	+0.08

Table 3: both ablations (paper TABLE III and TABLE IV merged), measured in the Realistic Dynamic Environment.

Runtime and staleness profile. This is the table that connects mechanism to outcome: on a single RTX 4090 with batch size 1, it reports the wall-clock latency of one forward pass (observation to action) and the pedestrian displacement accrued during that window, each with an average and a worst case. Because streaming exposes usable guidance from the first token rather than only at end of generation, SPARK-VLN (Stream) cuts average per-update latency from 0.788 s (W-t-A) to 0.185 s, the fastest among all VLM-based methods and an order below the similarly sized NaVILA (0.603 s), DualVLN (0.637 s) and video-based NaVid (0.741 s); Uni-NaVid at 0.193 s is comparable. Lower latency converts directly into lower staleness: average pedestrian displacement during one inference falls from 0.213 m to 0.050 m, and the worst case from 0.893 m to 0.694 m. Half a decimeter is well inside a single step, whereas NaVILA's 0.483 m is close to a full stride. In a frontal approach or an intersection, that difference is the difference between yielding and colliding.

Method	Backbone / params	Lat. Avg ↓ (s)	Lat. Max ↓ (s)	Disp. Avg ↓ (m)	Disp. Max ↓ (m)
NaVid	LLaMA-VID / 7B	0.741	2.864	0.394	1.864
Uni-NaVid	LLaMA-VID / 7B	0.193	0.795	0.202	0.773
NaVILA	VILA / 8B	0.603	1.140	0.483	1.514
DualVLN	Qwen-VL-2.5 / 7B	0.637	2.427	0.269	2.801
SPARK-VLN (W-t-A)	VILA / 8B	0.788	1.325	0.213	0.893
SPARK-VLN (Stream)	VILA / 8B	0.185	1.030	0.050	0.694

Table 4: runtime profile in the Realistic Dynamic Environment (paper TABLE V), single RTX 4090, batch size 1. The same VILA-8B backbone and the same flow-matching planner, with only the transfer mechanism changed, go from 0.788 s to 0.185 s. That 4.3x speedup is not bought by model compression; it comes from replacing "wait for generation to end" with "use the part already generated".

Fig. 6: radar comparison across tasks and environments. (a,b) VLN and (c,d) point-goal navigation, each under the idealized and the realistic dynamic environment. SPARK-VLN achieves the largest coverage in every setting, with raw values annotated. The gap between the idealized and realistic panels reflects each method's robustness to the observation staleness induced by inference latency.

Fig. 7: qualitative comparison in the Realistic Dynamic Environment. Each row shows the egocentric RGB view, the third-person view and the top-down occupancy map, where the blue curve is the executed trajectory toward the goal (red pin) and the orange curve is the reference ground-truth path computed without pedestrians. Scene 1 (vs. NaVILA): SPARK-VLN follows the instruction faithfully and proactively avoids the moving pedestrian, reaching the target near the couch, while NaVILA deviates from the instructed route and eventually collides. Scene 2 (vs. DualVLN): the dual-system baseline must finish full VLM inference before handing guidance to its fast planner, so the robot reacts late and collides with the pedestrian in gray; SPARK-VLN streams guidance during decoding and negotiates both pedestrians smoothly.

Qualitative evidence. These episodes translate the numbers of Table 1 into concrete behavior. The W-t-A baseline stalls during inference and then reacts abruptly, colliding with an oncoming pedestrian, whereas SPARK-VLN anticipates that pedestrian from progressively refined guidance and reaches the goal. It is the runtime table expressed in a different register: latency is not an abstract efficiency figure but the quantity that decides whether the robot has usable semantic guidance during the half second in which it meets a person.

Limitations

Stated by the authors 1: bounded instruction horizon. The paper explicitly lists extending to longer-horizon instructions with multiple sub-goals as future work. The benchmark's instruction vocabulary covers action, direction, place and object roles, and the median start-to-goal geodesic distance is only 8.2 m, so these are single-leg, short-range commands. Once an instruction becomes a multi-subgoal sequence ("go up to the second floor, cross the corridor, turn left at the second door, find the blue folder"), eight latent slots must simultaneously encode which sub-goal has been completed and where the next one is. Whether a fixed-capacity bridge becomes the bottleneck there is not tested.

Stated by the authors 2: reactive avoidance only. The authors also list incorporating pedestrian intent prediction for anticipatory rather than reactive social planning. In the current framework the safety critic $C_{\omega}$ scores collision-freeness and kinematic smoothness only, so pedestrians are treated as moving obstacles rather than intentional interaction partners; streaming shortens the perceive-to-react delay but does not make the robot infer where a person intends to go next. AvgD of 5.13 m and PSC of 93.9% were measured against ORCA pedestrians that actively avoid the robot themselves, and human pedestrians are considerably less predictable.

Our reading 1: the collision rate is still high. A 29.3% collision rate in the realistic stage means nearly one episode in three ends with contact with a pedestrian, an absolute level that is unacceptable in any real deployment framing. It is the best in the field, which shows that streaming is an effective lever, but there is a long distance between "relatively best" and "usable", and the paper does not discuss what it would take to close it. The idealized-stage figure of 24.8% deserves attention too: with no latency pressure, SPARK-VLN collides about twice as often as NaVid (11.8%) and CMA (13.8%), suggesting an aggressive policy style whose social-compliance advantage is driven mainly by the latency edge in the realistic stage.

Our reading 2: simulation only, and missing ablations. Every result comes from the simulated benchmark; there is no real-robot experiment, and the behavioral gap between ORCA pedestrians and real crowds would land directly on the collision rate. Several key design choices also lack ablations: why layers 16-23 rather than earlier, later, or all layers; why $N=8$ slots instead of 4 or 16; and what the accuracy-speed trade-off of $J=5$ Euler steps looks like. Only the implementation values are reported. The VLM is frozen as well, which means all the gain comes from the transfer mechanism rather than semantic capability, and it remains unknown whether the design still holds with a weaker backbone.

Summary and Outlook

The contributions separate into a method layer and an evaluation layer that make each other possible. On the method side, the paper rewrites the extract-project-infuse communication pipe of dual-system navigation into a token-level stream: the Token-Wise Hidden Streamer makes guidance available while the VLM is still reasoning, the Sequence-to-Slot Latent Bridge uses 8 learnable queries to compress a variable-length high-dimensional stream into a fixed-size condition the planner can consume, and the Evolving Latent Conditioner exploits a unified goal token so that the same flow-matching planner serves both point-goal navigation and language navigation. None of the three modules is a new component on its own, since intermediate hidden states, Perceiver-style cross-attention and rectified flow matching are all established techniques. What is new is assembling them into a conditioning path that refreshes continuously within a single inference pass, together with the quantitative claim that follows: changing only the delivery mechanism buys 10 SR points and a 4.3x latency reduction.

The evaluation layer may prove the more durable part. The asynchronous two-stage protocol promotes an implementation detail that has always been silently assumed, namely whether the simulator pauses while the model thinks, into a first-class experimental variable, and it reports explicit staleness statistics in the form of pedestrian displacement during inference. Once such a protocol is adopted, a family of methods that looked equivalent under blocking benchmarks gets separated. The paper's own numbers demonstrate this: NaVid ranks second in the idealized stage and loses nearly a third of its SR in the realistic one, and DualVLN, despite being a dual-system method, ends up with a realistic-stage collision rate of 46.6%, worse than most single-system baselines. The label "dual system" therefore guarantees nothing about robustness; what matters is when, and at what granularity, the two systems communicate.

Two lessons transfer to any team building real-time embodied systems. First, the output of a slow system need not be an atomic unit: intermediate representations from autoregressive decoding carry usable information, and treating them as a stream pays off better than treating them as a return value. That holds for any large-model-plus-fast-executor architecture, not just navigation. Second, the shape of the conditioning path has to match the calling frequency of the executor. A variable-length, high-dimensional semantic stream feeding a real-time planner that requires fixed-size, low-dimensional input needs an explicit compression bottleneck in between, and the difference between pooling and cross-attention at that position is 9.6 SR points and 7.2 collision-rate points. The two directions the authors put forward, multi-subgoal long-horizon instructions and pedestrian intent prediction, are precisely the next stress tests of these two lessons: longer instructions test the expressive capacity of fixed-size slots, while intent prediction pushes the channel from a reactive guidance stream toward a predictive interaction model.

Golden Quote

"The bottleneck, then, lies not in the reasoning process but in its delivery: guidance is withheld until reasoning completes, even though usable knowledge emerges token by token along the way." This sentence is the fulcrum of the whole paper. Accept it, and you go fix the communication pipe instead of the model capacity. Doing exactly that, with the same VILA-8B and the same flow-matching planner, changes delivery from "end of generation" to "every emitted token", and latency drops from 0.788 s to 0.185 s, success rate rises by 10 points, and during one inference the pedestrian has moved only five centimeters on average.