Introducing LightNav-0: Scaling Real2Sim2Real for Zero-Shot Generalist Navigation

Scaling Real2Sim2Real for zero-shot generalist navigation

Building embodied foundation models that generalize zero-shot across robot embodiments, tasks, and scenes requires post-training — or alignment — on large-scale real-world experience. Real-world teleoperation is the primary way such experience is collected today, but reaching the scale and diversity needed for zero-shot generalization remains difficult.

Light Origins addresses this with a Real2Sim2Real data engine that synthesizes simulation-based surrogates for real-world experience at scale. It turns 2,000+ internet-sourced real-world scenes into reusable simulated worlds, yielding 4,000+ hours of diverse vision-language-action (VLA) experience for navigation post-training.

That data trains a generalist navigation model through three stages: Embodied Reasoning (ER) mid-training, Embodied Supervised Fine-Tuning (SFT), and Online Reinforcement Learning (RL). The result is LightNav-0, a single model that achieves state-of-the-art performance across 10 simulation settings spanning instruction following, object-goal navigation, and embodied visual tracking — and transfers zero-shot to humanoid, quadruped, aerial, and wheeled robots.

Light Origins frames its path toward Physical AGI as three scaling paradigms: scalable pre-training, scalable alignment, and scalable deployment. LightNav-0 is their first step toward scalable alignment through Real2Sim2Real, and the claim on display is that alignment at scale is the key to zero-shot generalization in the physical world.

01 · Breaking the embodied post-training cold start

Embodied post-training has a production problem. Physical experience arrives one robot, one environment, and one attempt at a time, while generalization depends on combinations of bodies, worlds, and tasks. Collecting those combinations in the real world is not only expensive — it is too slow for the iteration cycle that modern model training requires.

Scalable Alignment starts with 2,000+ internet-sourced real-world scenes. Rather than treating them as static assets, the data engine brings different scene formats into one simulation interface, where each world can support many goals, routes, viewpoints, and task formulations. Camera geometry varies with the experience, so the training distribution is not tied to one field of view, mounting height, or robot body.

Each rollout becomes one aligned record of what the model saw, what the instruction referred to, where it should attend, and how it should move. That alignment — not rendering volume alone — is what turns scene scale into post-training value. The current engine produces 4,000+ hours of vision-language-action experience; physical deployment then reveals which scenes, tasks, and interactions the next cycle should add.

02 · The post-training recipe

Embodied foundation models increasingly follow a familiar post-training recipe: domain mid-training, supervised fine-tuning, and reinforcement learning. The progression is deliberate — mid-training shapes the representation, SFT teaches the behavior, and RL improves outcomes under the policy's own distribution. LightNav-0 adapts this recipe to navigation through ER mid-training, Embodied SFT, and Online RL.

02A · Stage I — spatial reasoning before action alignment

Robot trajectories are expensive because each must align a scene, an instruction, and a sequence of actions. Yet much of what a navigation policy needs before acting — grounding language in pixels, locating free space, reasoning about spatial relations, and tracking change — can be learned from images and video at far greater scale. Stage I builds these spatial priors during mid-training; Stage II uses trajectories to connect them to action.

This pattern is becoming standard: Gemini Robotics is built on Gemini Robotics-ER, while MolmoAct2 uses MolmoER as its VLM backbone — two examples of embodied reasoning becoming the foundation for action learning. LightNav-ER plays that role for LightNav-0 by initializing Stage II with spatial priors. Before action alignment, it ranks first on four and second on four of the eight embodied-reasoning benchmarks reported, with a macro-average of 67.4 — +4.3 over its Qwen3-VL initialization.

Benchmark	Point-Bench	RefSpatial	RoboSpatial-POI	RoboSpatial-VQA	Where2Place	CV-Bench	ERQA	EmbSpatial
LightNav-ER	64.5	57.4	56.5	71.9	76.6	88.4	43.8	79.8

Fig. 03 data — Stage I strengthens spatial reasoning before action training: 67.4 average, 4 first places, 4 second places, +4.3 over initialization. The benchmark suite covers Point-Bench, RefSpatial, RoboSpatial (POI and VQA tracks), Where2Place, CV-Bench, ERQA, and EmbSpatial.

The Stage I mixture is dominated by active sampling, with image pointing as the largest specialized component:

Capability family	Share of Stage I mixture
Image pointing	35.14%
Single-image VQA	25.05%
General visual & abstract reasoning	20.00%
Video reasoning	19.81%

Fig. 02 data — What Stage I trains on: pointing accounts for 35.14% of the mixture across 36 sources; visual, abstract, and temporal reasoning make up the remaining 64.86%.

Entity grounding

A core capability that carries over from Stage I: grounded phrases stay linked to their image-space points. Asked to "list the home appliances in this image and point out where each one is," the model returns each phrase with coordinates — refrigerator, water dispenser, microwave, coffee makers.

02B · Stage II — building embodied SFT data

Stage I gives the model a spatial prior; Stage II has to turn it into behavior. Physical collection cannot cover enough combinations of scenes, goals, routes, and viewpoints at post-training scale. Real2Sim2Real changes the unit of scale: capture a scene once, then use it to produce many executable interactions.

The pipeline runs real → simulation → aligned data → post-training → real, in four automatic steps:

1. Target annotation (Molmo2 annotates open-set objects). Across every scene source, image-space points are lifted through metric depth into one target inventory. For unlabeled Gaussian splats, Molmo2 points to candidate objects in rendered views; the engine lifts those points into 3D and keeps only targets that agree across viewpoints (≥2 views per instance, spread < 0.6 m). Scenes with annotated 3D boxes enter directly.
2. Episode synthesis (the simulator generates executable rollouts). It samples a target and a visible start, plans and anchors the route on the navigation mesh, and renders egocentric video with action labels — at scale, many routes per scene.
3. Verified language (VLMs turn the rollout into verified language). Template or video-VLM drafts are rewritten under semantic checks, then verified against the final views.
4. One rollout → one consistent multimodal record.

This order matters: the instruction describes behavior that exists, rather than asking a planner to recover behavior from prose written in isolation. The action trajectory is generated first; language is added without being allowed to change its target or geometry. Language is the uncertain component, so the engine generates it along two independent paths and verifies it twice — a geometric path (goal pixel plus start-relative bearing across eight directional sectors produces a target-aware template) and a video path (Seed2.0 drafts a video-grounded instruction from ordered route frames), converged on one language surface. This is what turns a rendered rollout into an aligned training example.

INSIGHT-Bench documents one reproducible slice of this engine: 53,090 training episodes from 1,683 scenes, plus a frozen evaluation split of 1,097 episodes from 210 scenes. The scenes mix conventional meshes with Gaussian-splatting reconstructions; the episodes mix five kinds of spaces with five distinct ways an instruction can identify its goal.

Training episodes by scene source	Episodes	Share
HM3D / MP3D	34,531	65.0%
InteriorGS	8,534	16.1%
HabitatGS	5,776	10.9%
VLNVerse	4,249	8.0%

The renderer also treats sensor geometry as a data axis. Within the same scene, it samples a 90–130° field of view, a camera height of 0.5–1.5 m, and a pitch of −15° to 15°. This exposes the policy to body-scale viewpoint changes during training instead of binding every route to one camera rig.

02C · Stage II — Making spatial reasoning executable

These aligned records are where Stage I's spatial prior becomes behavior. From the same language goal and visual history, the model learns three linked decisions: what the goal refers to, where motion is possible, and which trajectory reaches it. Because all three are supervised by the same interaction, Stage II learns perception and action as one problem rather than separate tasks.

The training mix preserves that foundation: most examples teach navigation and action, while a smaller reasoning stream keeps Stage I's spatial capabilities active. With Stage II data held fixed, LightNav-ER improves every evaluated setting. A compact interface carries this prior into behavior — image-space Point CoT exposes spatial intent, while RVQ motion tokens express the trajectory that follows.

The full Stage II mixture goes beyond this documented slice: sixteen navigation sources are co-trained with 33 reasoning sources. R2R, RxR, ScaleVLN, and SRDF enter the stream alongside tracking data, outdoor scenes, and Gaussian-splatting worlds. The instructions remain long-form: 742K total, 474K unique, with a median length of 20 words.

The Stage II training mix

Available data pool	Total examples
VQA	5.2M
ObjectNav	4.7M
SRDF	4.7M
ScaleVLN	2.8M
Tracking	2.8M
VLN-CE · RxR	1.8M
VLN-CE · R2R	642K
INSIGHT-Bench	454K

During Stage II, 77.6% of sampled examples teach navigation and action; VQA accounts for the remaining 22.4%.

ER improves all eight settings

Fig. 06 — with identical Stage II data, ER initialization raises mean SR from 60.8 to 63.1 and mean SPL from 39.0 to 40.0, improving all eight settings (best gain +4.2 on MP3D):

Setting	Gain from ER initialization
R2R	+2.7
RxR	+1.0
HM3D v1	+3.1
HM3D v2	+1.5
MP3D	+4.2
OVON Seen	+1.6
OVON Synonyms	+0.8
OVON Unseen	+4.0

Entity grounding

Grounded phrases stay linked to their image-space points. In the example below, the model is asked to list the home appliances in an image and point out where each one is; every named entity is anchored to pixel coordinates.

Routes teach where to look and move

Because the Real2Sim2Real engine retains the referred object, camera pose, and route for every frame, each episode also supplies its own point labels. A reachable future waypoint projects to an affordance point; whenever the referred object enters view, its 3D target projects to an object point. Point supervision therefore grows with the generated routes — without a separate annotation pass.

Fig. 07 — simulation state becomes point supervision by projection. The object point and affordance point remain attached to the frame and route from which they were derived.

Point CoT: why image-space points beat text and 3D coordinates

Point CoT began with a constraint: reasoning for autoregressive control must be cheap enough to run at every step and structured enough to supervise at data-engine scale. Textual CoT was considered first — expressive, but variable in length, costly to decode, difficult to synthesize consistently, and hard to falsify when an action fails. Metric 3D targets were also considered — precise, but they tie supervision to depth, pose, camera calibration, reconstruction, and a common coordinate system across embodiments.

Image-space points offered a simpler alternative. Each decision adds only two point tokens: an object point grounds the referred object when it is visible, and an affordance point identifies free space for progress. Simulation geometry supplies both labels at scale. At deployment, the model predicts them from RGB and language — without depth, localization, calibration, or a map. Because the format is fixed, the reasoning sequence stays short.

With everything else held fixed, Point CoT improves all eight settings, raising mean SR by +8.4 and mean SPL by +5.7:

Setting	SR gain with Point CoT
R2R	+9.0
RxR	+2.8
HM3D v1	+13.5
HM3D v2	+11.6
MP3D	+12.0
OVON Seen	+7.1
OVON Synonyms	+7.5
OVON Unseen	+3.9

Motion as tokens

Each training target is a metric SE(2) trajectory with ten waypoints. To place motion in the same autoregressive interface as language and image-space points, Light Origins learns a residual vector quantizer over the Stage II trajectory distribution.

The tokenizer comprises three 256-entry codebooks. L0 selects a coarse trajectory; L1 and L2 encode successive residuals. The resulting three-token representation achieves a mean waypoint displacement error of 0.72 cm. Ablation supports three levels as the minimal sufficient hierarchy: two levels incur substantially higher reconstruction error, whereas additional levels or larger codebooks yield only marginal reductions. Every prefix remains decodable — L0 defines a complete coarse trajectory, and each subsequent token refines it.

Environment scaling has yet to saturate

Because the prediction interface and Stage II recipe remain unchanged, scale can vary along three axes without changing the learning objective: the number of training environments, the number of aligned SFT examples, and model capacity. Across the tested range, widening the set of environments is the most reliable way to improve the policy — all four measures rise at every step of environment scaling. Increasing model size produces mixed results, while additional SFT episodes from the same scene pool show diminishing returns. At the current scale, adding new environments is likely to be more valuable than sampling existing ones more densely.

Real2Sim2Real makes that expansion practical: each captured environment is reconstructed once, then reused to produce many aligned goals, routes, instructions, and camera views. More compute raises episode throughput; more scene assets broaden the visual and spatial variation those episodes cover.

02D · Stage III — Train beyond demonstrations

Even broad SFT data has a boundary: it shows the policy how successful navigation unfolds, but not how to recover from its own mistakes. A late turn, an overshoot, or a lost target takes the policy into states that demonstrations rarely cover. Stage III keeps the same point-and-action interface but learns from a new source of experience — the policy's own trajectories and their outcomes.

Stage III remains entirely in simulation. Instead of following routes supplied by the data engine, the model proposes complete plans and the simulator scores their outcomes. The policy's own states — and its own mistakes — now shape the training signal. GRPO learns from the relative outcomes of those plans. The action vocabulary stays fixed — RL operates on the same RVQ motion tokens introduced during SFT — while supervision shifts from imitating a successful route to improving the consequences of the policy's own decisions.

The distracted-tracking split of EVT-Bench provides a consistent readout: success rate rises from 74.4 at the SFT checkpoint to a best observed value of 82.6 at step 120, evaluated over 1,392 episodes in 101 scenes, before reaching a plateau.

03 · Generalization across tasks, scenes, and embodiments

Across three stages, the model learns from broad spatial reasoning, aligned simulated interactions, and simulated outcomes from its own policy. The training loop ends there. What follows is transfer — across public benchmarks, physical embodiments, and unfamiliar visual domains.

Public benchmarks are the first transfer test. The same checkpoint is evaluated across ten settings spanning instruction following, object navigation, open-vocabulary search, and target tracking, using VLN-CE, Matterport3D and HM3D, HM3D-OVON, and EVT-Bench.

LightNav-0 uses a single egocentric RGB view, limited to the camera's current field of view. Panoramic and multi-camera methods observe several directions at once. The table therefore separates monocular methods under the same sensing constraint from broader-view systems:

Setting	Metric	LightNav-0 (single RGB)	Best monocular rival	Best broader-view system
VLN-CE · R2R val-unseen	SR ↑	68.5 (#1 mono)	Qwen-RobotNav-4B · 66.9	Qwen-RobotNav-8B · 72.1
VLN-CE · RxR	SR ↑	73.6 (#1 mono)	Qwen-RobotNav-8B · 73.4	Qwen-RobotNav-8B · 76.5
MP3D object-goal nav	SR ↑	53.3 (#1 mono)	CogNav · 46.6	Qwen-RobotNav-4B · 52.2
HM3D v1 object-goal nav	SR ↑	74.5 (#1 mono)	Uni-NaVid · 73.7	WMNav · 58.1
HM3D v2 object-goal nav	SR ↑	79.5 (#1 mono)	FiLM-Nav · 77.0	Qwen-RobotNav-4B · 75.6
HM3D-OVON · Seen	SR ↑	55.3 (#1 mono)	MTU3D · 55.0	Qwen-RobotNav-4B · 57.7
HM3D-OVON · Synonyms	SR ↑	53.3 (#1 mono)	MTU3D · 45.0	Qwen-RobotNav-4B · 60.1
HM3D-OVON · Unseen	SR ↑	47.0 (#1 mono)	MTU3D · 40.8	ABot-N0 · 54.0
EVT-Bench · Single-target tracking	SR ↑	91.7 (#1 mono)	ReferTrack · 89.4	CoMaTrack · 92.1
EVT-Bench · Distracted tracking	SR ↑	82.6 (#1 mono)	ReferTrack · 73.3	CoMaTrack · 74.2

LightNav-0 ranks first in all ten monocular comparisons. When broader sensing is included, it remains first in four.

Rethinking navigation evaluation

Most navigation benchmarks compress several abilities into one success rate. They are also concentrated in Habitat-style indoor scans and often use long route descriptions. When a model fails, the score does not reveal whether it misunderstood the goal, missed a spatial relation, or could not navigate the scene.

Light Origins therefore built INSIGHT-Bench to separate those factors. It evaluates five atomic ways of specifying a goal — Base, Direction, Relation, Extremum, and Ordinal — across apartments, houses, commercial spaces, institutions, and outdoor environments. The benchmark contains 1,097 episodes in 210 held-out scenes. Among seven open-source policies evaluated under the same forward-RGB protocol, LightNav-0 achieves the highest success rate in every instruction and scene category.

Policy	Base	Direction	Relation	Extremum	Ordinal
LightNav-0	45.1	57.7	37.8	37.2	38.2
NaVid	37.6	29.7	18.7	22.4	24.2
JanusVLN	32.5	28.5	28.5	22.0	25.8
Uni-NaVid	32.9	29.3	21.8	16.8	19.7
TAMP-Nav	18.1	14.2	17.6	10.8	20.8
StreamVLN	15.6	9.6	14.5	8.0	10.7
InternVLA-N1	18.1	13.0	9.8	8.4	7.9

Policy	Apartment	House	Commercial	Institution	Outdoor
LightNav-0	61.1	50.5	42.1	29.2	34.2
NaVid	37.7	39.8	24.6	18.3	13.6
JanusVLN	39.7	35.2	26.7	18.3	16.7
Uni-NaVid	39.3	28.7	23.1	15.5	14.0
TAMP-Nav	26.8	18.5	15.9	9.6	8.3
StreamVLN	19.7	15.7	12.8	4.1	5.3
InternVLA-N1	12.1	19.0	12.3	7.8	7.5

Zero-shot transfer is the test of embodied post-training

A policy can be valuable on a single robot. Generalization is the stronger test: what remains when the body, environment, or task changes. LightNav-0 is evaluated along all three axes — across four physical robot embodiments, unseen indoor and outdoor environments, and three task families: long-form instruction following, open-vocabulary object navigation, and target tracking.

04 · Beyond embodied navigation

Scaling data matters only if the resulting policy works beyond the worlds that produced it. The unchanged checkpoint is therefore tested far outside the visual distribution of its robot training data: first in closed-loop interactive worlds, then on open-loop in-the-wild video.

The two settings answer different questions. In interactive worlds, every predicted action changes the next observation, so errors accumulate and the policy is tested end to end. In video, the model cannot influence what it sees next — this isolates whether target grounding, free-space reasoning, and trajectory prediction survive unfamiliar cameras, motion, and visual styles.

Closed-loop control in unfamiliar worlds

Each environment adds a different challenge. The voxel world changes scene geometry and texture. The driving world demands continuous steering at speed, leaving less time to correct drift. The first-person worlds introduce legacy graphics, and the final one also changes both resolution and aspect ratio with a lower-resolution 4:3 frame.

Open-loop visual transfer

Mountain biking, speed skating, freestyle skiing, and trail running introduce rapid egomotion, occlusion, scale change, and non-robot camera dynamics absent from the embodied post-training data. Across these videos, LightNav-0 continues to ground the target and predict feasible motion frame by frame.

05 · LightNav-0 is open

The release brings together the model, code, and technical report behind LightNav-0, with a broader evaluation suite to follow. Light Origins thanks its collaborators at LimX Dynamics and Manycore Tech for their support across physical deployment and simulation.

---

Source: Introducing LightNav-0 — Scaling Real2Sim2Real for Zero-Shot Generalist Navigation, Light Origins, Sept. 1, 2026. This article is a bilingual adaptation of the original post; figures, tables, and videos are reproduced from the source.