ASENA: Self-evolving Agents for Embodied Navigation

ASENA: Self-evolving Agents for Embodied Navigation
An-Chieh Cheng, Isabella Liu, Edmund Bu, Johan Bjorck, Hongxu Yin, Zhengyi Luo, Jan Kautz, Linxi "Jim" Fan, Yuke Zhu, Sifei Liu (NVIDIA; Edmund Bu at UC San Diego; Yuke Zhu also at UT Austin)
arXiv:2609.39207v2 [cs.RO], v2 updated 2026-10-12 · arXiv:2609.39207 · Project page: research.nvidia.com/labs/gear/asena
Code status: the project page provides demo videos, interface documentation, and inline code snippets (e.g. climb.py, onsurface.py), but as of this writing (2026-10-15) it contains no github.com link at all; code, weights, and the atomic navigation dataset are not publicly released, and the page's Dataset entry is only an in-page anchor (#training-data). The reproducibility anchors of the paper are therefore the protocol descriptions in the appendix: training sampling (64 devices, batch 512), pool composition (Table 7), the Jetson Thor quantization and timing protocol (A.2), and the G1 sensor and navigation parameters (A.5).

One-sentence summary

ASENA hands a humanoid robot to a coding agent as a programmable environment: the agent reads sensors, writes and executes programs, inspects recorded execution evidence to repair failures, and retains reusable notes and executable skills in a persistent workspace, all without updating any model weights; the optional 4B monocular navigation policy ASENA-VLN reaches state-of-the-art 68.7% / 70.2% success on R2R / RxR standalone, and as a tool inside the agent lifts a weaker coding backend by 11 success points while cutting mean tool calls per episode by up to 62.5%; over ten passes through recurring 100-task subsets, workspace evolution raises R2R success from 72% to 98% and RxR from 65% to 89%.

1. The problem: navigation is one step inside a larger task

The paper opens with a concrete request: check whether a vending machine contains an item and come back with an answer. To do it, the robot must decide where to search, recognize the item, remember where the user is standing, and communicate what it found; swap in retrieving an object from a room and it must also plan a grasp, pick the object up, carry it back, and hand it over. Navigation is therefore only one part of a larger task whose next step depends on what the robot just observed. A learned navigation policy supplies the capability to move, but such requests also require deciding what to inspect, how to act on objects, and what to communicate; these procedures depend on new observations and may extend beyond predefined skills.

Coding agents are exactly good at this: they can construct and revise procedures during a mission through programmatic access to sensing, computation, simulation, and action. What physical deployment still lacks is feedback tied to outcomes and a persistent workspace for retaining useful procedures. ASENA supplies both. The paper makes three contributions: a unified robot harness that lets coding agents program humanoid navigation, manipulation, and interaction while retaining executable skills and experience; ASENA-VLN, a state-of-the-art monocular navigation policy that supports both route following and reusable atomic skills; and a systematic evaluation showing that workspace evolution improves task performance while all model weights remain fixed. Evaluation covers navigation and embodied question answering (EQA) in simulation, plus a real Unitree G1 operating without a pre-built map.

2. System overview: one unified robot harness

ASENA folds sensing, computation, visualization, simulation, and action into a single write-run-inspect workflow. On the G1 the exposed sensing interface includes a mast-mounted 360-degree panoramic camera (Insta360 X5 or Ricoh Theta X, configured at 6 Hz JPEG with 180-degree yaw correction), an inverted Livox Mid-360 LiDAR inside the head shell (SLAM bridge at 10 Hz, at most 20,000 points per batch), odometry, a body IMU, 29 joint positions and velocities published at 20 Hz, and a calibrated head OAK RGB/stereo camera for manipulation. Body motion is executed by the SONIC whole-body controller; optional YAM parallel grippers with fin-ray fingers use a separate controller that reports jaw width and controller state; speech uses the onboard speaker.

Figure 1 (paper Figure 1): ASENA connects coding agents to physical robot skills. Programs combine sensing, computation, robot execution, and reusable skills, with learned navigation as an optional tool; the code example uses LiDAR geometry to propose a passage route, and the gallery shows navigation, inspection, grasp planning, object transport, handoff, and gestures across recorded demonstrations.

Figure 2 (paper Figure 2): system structure. Robot sensing, a persistent workspace, navigation and manipulation tools, and robot execution are integrated into one interface; body motion uses SONIC and optional YAM grippers use a separate controller. Every run is stored as an MCAP record aligning sensor and robot states with actions, feedback, code, and visualizations for inspection and replay; the agent reflects on these records to update its notes and skills.

3. Grounding programs in robot feedback: navigation, manipulation, execution checks

Navigation. The agent has two routes: geometric tools, or invoking ASENA-VLN for language-conditioned navigation. Geometric tools plan waypoints over LiDAR-based occupancy maps and check obstacle clearance; ASENA-VLN predicts bounded body-frame trajectories from language and recent monocular observations (Section 5). The agent itself chooses when to invoke each tool, checks progress through new observations, and revises the plan as needed; this "tool selection stays with the agent" design runs through the whole paper.

Manipulation. Grasp candidates are grounded by SAM 2 object masks plus depth measurements; when more options are needed the system supplements geometric proposals with learned candidates from GraspGenX. Inverse kinematics and motion planning convert these candidates into trajectories for reaching, lifting, and carrying. Before execution the system rebuilds arm trajectories from measured joint states and checks joint limits, sampled self-collisions, and scene clearance, including held-object clearance during lift and retraction. On the gripper side, a separate controller opens and closes the optional YAM parallel grippers and reports jaw width and controller state, so the agent can compose these tools with navigation to program pickup and handoff.

Execution checks. Generated gestures are validated online in MuJoCo for pose continuity, joint and motion limits, and sampled self-collision; failed validation returns the violated constraint to the agent for repair. Physical motion requires validation plus operator approval, including motion requested by revised programs. An independent navigation safeguard enforces motion limits and rejects commands when odometry is stale or the battery level is low; heartbeat-triggered stops and manual emergency halts add further protection. This layer is the gate between "the agent may write any program" and "the robot must not move arbitrarily".

4. Turning execution into retained experience: workspace evolution

Every run is recorded in MCAP format, aligning sensor and robot states with actions, feedback, code, and visualizations. Replaying this evidence lets the agent identify where execution diverged from intent and repair the responsible program. After repairing a failure, ASENA can retain the solution for future tasks: the workspace at revision $k$ is $W_k=(N_k,S_k)$, where $N_k$ holds notes and $S_k$ holds executable skills; a fixed coding model consults this workspace while solving tasks; between revisions, a separate fixed-weight improver updates the workspace from execution traces $\mathcal{T}_k$ and feedback $f_k$:

$$W_k=(N_k,S_k),\qquad W_{k+1}=I_{\phi}\bigl(W_k,\,\mathcal{T}_k,\,f_k\bigr) \tag{1}$$

Model and controller weights remain fixed throughout. In simulation, private workers may edit their workspaces between passes while writes are disabled during frozen evaluation; on the G1, execution records and optional operator feedback support skill revision while robot motion is disabled. In other words, all of the "learning" happens in text and code, not in gradients.

flowchart TD
    S["Sensing: 360-deg camera / Livox Mid-360 / odometry / IMU / joint states / head OAK camera"] --> A["Coding agent: read observations, estimate geometry, inspect visualizations, write programs"]
    A --> V{"Language-conditioned navigation needed?"}
    V -->|yes, optional tool| P["ASENA-VLN-4B: language + up to 8 monocular frames -> body-frame waypoints"]
    V -->|no| G["Geometric tools: LiDAR occupancy map waypoints with clearance checks"]
    P --> E["Execution: SONIC whole-body control + separate YAM gripper controller + onboard speaker"]
    G --> E
    E --> C["Pre-execution checks: MuJoCo gesture validation / joint limits / self-collision / clearance + operator approval + navigation safeguard"]
    C --> R["MCAP record: sensor and robot states aligned with actions, feedback, code, visualizations"]
    R --> Q{"Execution matched intent?"}
    Q -->|no| F["Replay evidence, locate divergence, revise program"]
    F --> A
    Q -->|yes| K["Retain notes and executable skills in persistent workspace W_k"]
    K --> I["Improver aggregates traces T_k and feedback f_k, updates workspace, weights frozen"]
    I --> A

5. ASENA-VLN: a 4B monocular navigation policy as a callable tool

The motivation for ASENA-VLN is direct: delegating motion to a specialized policy reduces repeated agent decisions and supports navigation behaviors that are difficult to specify geometrically. It extends Qwen3-VL-4B-Instruct with dedicated token vocabularies for trajectories and pixel goals. On the input side the model takes up to eight front-view frames and a structured JSON instruction; recency-weighted visual tokens preserve temporal context while emphasizing the latest observation. The instruction gives a concise navigation command and can optionally include a desired end pose, anchor objects, or motion constraints, which makes the policy naturally a callable tool: a program passes JSON in and gets a trajectory out.

Figure 3 (paper Figure 3): left, the navigation policy architecture: language and monocular history enter a shared autoregressive decoder, separate token blocks encode trajectories and pixel goals, visual QA keeps the original text tokens, and motion tokens decode to body-frame waypoints (architecture glyphs are schematic). Right, recorded training observations with instructions and overlaid trajectories; more examples appear in paper Figure 6.

Output representations. A shared autoregressive decoder predicts up to eight cumulative body-frame waypoints, each represented as $(x,y,\theta)$; fixed bins along each axis convert trajectory tokens into explicit motion targets without a separate continuous action head. Pixel-goal tokens provide auxiliary supervision by encouraging the model to localize its intended destination in the image, but a pixel goal is not directly executable: it needs an additional motion planner (or, in simulation, a shortest-path follower), so every evaluation in the paper uses trajectory-token predictions rather than pixel goals. The added-token ledger is 660 trajectory tokens plus 683 pixel-goal tokens, 1,343 in total.

$$\tau=\bigl[(x_i,\,y_i,\,\theta_i)\bigr]_{i=1}^{m},\qquad m\le 8 \tag{2}$$

6. Atomic navigation data and the training mixture: teaching reusable fine-grained motions

A callable navigation policy must follow extended routes and precise local commands alike. The paper therefore curates nine atomic navigation tasks in MP3D and HM3D, covering metric motion (move a specified distance), pure rotation (rotate to a target heading), heading-relative goals (approach an object from a given direction), fine alignment, entering a room, passing through a doorway, following a corridor, and moving between floors. Each task is generated and validated using geodesic distance, visibility, and scene geometry so that every instruction describes a feasible and measurable motion; each instruction is paired with a waypoint trajectory and auxiliary pixel-goal supervision. These data directly teach the reusable, fine-grained operations that task programs can invoke and compose.

Figure 4 (one example from paper Figure 6): an atomic navigation training sample. The observation carries an overlaid waypoint trajectory for an instruction such as "step into the kitchen through the door"; the dataset spans metric motion, rotation, relative goals, alignment, room goals, doors, corridors, and floor changes.

Training mixture. ASENA-VLN trains on these atomic tasks, short-horizon ObjectNav, and vision-language navigation data from R2R, RxR, ScaleVLN, and SRDF, plus multi-view VQA data to strengthen spatial understanding. To match the callable-policy interface, original navigation instructions are re-captioned by GPT-5.5 into the structured JSON format, including concise commands and, when applicable, end poses, anchor objects, and motion constraints. Pool sizes appear in paper Table 7: about 28.4M for the trajectory pool (including repeats) and about 9.1M for the VQA pool. Training uses a 3:1 mixture of trajectory prediction and spatial VQA; to emphasize stopping behavior, five extra copies of each eligible trajectory-ending window are added.

Sampling ledger. Training uses 64 devices and an effective batch size of 512: each device processes three trajectory examples and one VQA example per step, with gradients accumulated over two steps:

$$512=64\times(3+1)\times 2 \tag{3}$$

The reported checkpoint uses fewer than 10K optimizer steps and fewer than 5M training samples including repeats. For a 4B model this is a deliberately modest recipe, and the paper treats it as part of the point: the gains come from atomic data and interface design, not from stacked compute.

7. Standalone navigation: new SOTA on R2R and RxR

Table 1 evaluates ASENA-VLN independently of coding-agent planning on R2R's shorter instructions and RxR's longer, more detailed routes, with 0.25 m forward steps and 15-degree rotations. On metrics: NE is the final goal distance in meters, OSR measures entering the goal region, SR additionally requires stopping there, SPL accounts for path efficiency, and nDTW measures reference-route similarity, the last four in percentages; SPF denotes the simulator shortest-path follower used by pixel-goal methods.

Method	R2R NE↓	R2R OSR↑	R2R SR↑	R2R SPL↑	RxR NE↓	RxR nDTW↑	RxR SR↑	RxR SPL↑
NaVid	5.72	49.2	41.9	36.5	5.72	–	45.7	38.2
NaVILA	5.22	62.5	54.0	49.0	6.77	58.8	49.3	44.0
StreamVLN	4.98	64.2	56.9	51.9	6.22	61.9	52.9	46.0
NaVIDA	4.32	69.5	61.4	54.7	5.23	67.0	57.4	49.6
DualVLN	4.05	70.7	64.3	58.5	4.58	70.0	61.4	51.8
Qwen-RobotNav-4B	4.22	73.6	66.9	60.5	4.15	68.6	71.3	61.5
ASENA-VLN-4B (ours)	3.52	72.6	68.7	64.2	3.90	73.1	70.2	59.7

ASENA-VLN-4B takes the best R2R val_unseen result with 3.52 m NE and 68.7% SR, 1.8 points above the strongest same-scale baseline Qwen-RobotNav-4B (66.9%); on RxR its 70.2% SR and 73.1 nDTW also lead, at the cost of a slightly lower SPL than Qwen-RobotNav-4B's 61.5 (59.7). The OSR-versus-SR gap is worth noting: 72.6 against 68.7, meaning the stopping decision still costs about four success points, which is exactly what the five extra copies of trajectory-ending windows are meant to squeeze.

8. Object-goal navigation: global search reasoning paired with learned local navigation

Object-goal navigation requires finding an object that may not initially be visible. ASENA-VLN is trained primarily for atomic navigation behaviors rather than global exploration, so the paper pairs it with a separate VLM that plans where to search and updates the plan from new observations. On HM3D-v2 and HM3D-OVON this combination of global search reasoning and learned local navigation achieves state-of-the-art results against published methods, and performance stays consistent across seen, synonymous, and unseen object categories, indicating the system handles variation in both object descriptions and the routes required to find them.

Method	HM3D v2 val (SR/SPL)	OVON seen	OVON syn.	OVON unseen
VLFM	63.6/32.5	35.2/18.6	32.4/17.3	35.2/19.6
Uni-NaVid‡	73.7/37.1	41.3/21.1	43.9/21.8	39.5/19.8
ABot-N0	–	55.3/32.1	55.4/33.2	54.0/30.5
ApexNav	76.2/38.0	–	–	–
Qwen-RobotNav-4B	75.6/30.6	57.7/24.4	60.1/25.1	53.1/20.9
ASENA (ours)	80.8/38.1	60.5/33.5	65.8/37.9	64.9/38.4

‡ denotes HM3D v1 results, which the paper excludes from emphasis. ASENA beats ApexNav by 4.6 SR points on v2 val and leads Qwen-RobotNav-4B by roughly 3 to 6 points across all three OVON splits, with SPL improving in tandem: the accuracy is not bought with extra walking.

9. Complementary agent and policy capabilities: who actually needs learned navigation

Table 3 is the most information-dense table in the paper: it places published zero-shot methods, standalone ASENA-VLN, and ASENA instantiated with two coding backends in the same coordinate system. The R2R and RxR Agentic Splits each contain 100 tasks, following the reduced-set evaluation practice of Open-Nav and Uni-LaViRA; all policy and agent variants use the same task IDs across ten scenes per benchmark; four RxR episodes produce non-finite NE or SPL and count as failures for SR, with NE and SPL computed over the remaining 96. Every coding agent starts from a fresh workspace; Claude Sonnet 5 runs at high effort and GPT-6 Astra at maximum effort.

Method / coding agent	VLN tool	R2R NE↓	R2R OSR↑	R2R SR↑	R2R SPL↑	R2R calls/ep.↓	RxR NE↓	RxR SR↑	RxR SPL↑	RxR calls/ep.↓
InstructNav	–	6.89	47.0	31.0	24.0	–	–	–	–	–
Uni-LaViRA	–	3.66	73.7	60.7	47.7	–	6.48	51.3	34.0	–
ASENA-VLN-4B standalone	–	3.16	82.0	78.0	70.7	–	4.54	62.0	52.9	–
ASENA (Claude Sonnet 5)	×	5.23	70.0	61.0	34.2	83.32	5.80	50.0	31.9	103.83
ASENA (Claude Sonnet 5)	✓	3.31	83.0	72.0	54.9	33.96 (−59.2%)	4.94	61.0	45.5	38.91 (−62.5%)
ASENA (GPT-6 Astra)	×	1.84	90.0	87.0	72.4	26.37	0.82	93.0	78.0	40.80
ASENA (GPT-6 Astra)	✓	2.18	89.0	88.0	72.8	14.00 (−46.9%)	1.65	86.0	64.0	19.98 (−51.0%)

Three conclusions hold separately and do not hide each other. First, strong coding agents navigate well without a learned policy: with only geometric and programmatic tools, GPT-6 Astra reaches 87.0% SR on R2R, above standalone ASENA-VLN (78.0%) and close to the reported human performance of 90% SR; its reasoning suffices to interpret instructions, construct routes, monitor progress, and recover from errors, which establishes coding agents themselves as strong zero-shot navigation systems. Second, learned navigation helps weaker backends most: for Claude Sonnet 5, adding the policy raises SR by 11 points on both R2R and RxR (61.0→72.0, 50.0→61.0) with large SPL gains, compensating for its route-execution limitations; GPT-6 Astra, already strong, benefits less, and on RxR enabling the policy slightly reduces its SR from 93.0 to 86.0, with the paper stating plainly that the value of learned navigation depends on the coding backend and on how effectively it incorporates policy predictions into its plans. Third, call counts fall regardless of what success does: a single ASENA-VLN call executes several motion decisions, reducing repeated observation and action requests, and mean tool calls drop for every backend and benchmark:

$$1-\frac{33.96}{83.32}=59.2\%,\qquad 1-\frac{38.91}{103.83}=62.5\% \tag{4}$$

(Claude Sonnet 5 on R2R / RxR; GPT-6 Astra sees 46.9% / 51.0%.) Even where success barely moves, the policy absorbs low-level navigation, which is the real selling point of "learned policy as a tool" versus "learned policy as the brain".

10. Workspace evolution: ten passes over recurring tasks, 72% to 98%

The evolution evaluation repeatedly presents the same tasks, scenes, instructions, and initial poses. In each pass, sixteen high-effort Claude Sonnet 5 workers execute the tasks and may retain programs, notes, and prior action sequences in private workspaces; between passes, an improver uses their traces and feedback to update the canonical workspace. Campaigns run with and without ASENA-VLN on R2R and RxR using ground-truth feedback, while all model, policy, and controller weights stay frozen.

Figure 5 (paper Figure 4): workspace self-evolution on the R2R and RxR Agentic Splits (100 tasks each). Each pass revisits the same tasks with ground-truth feedback; workspace experience is retained while all model weights remain fixed. Success-related metrics improve, logged cost per episode generally decreases, and the VLN tool consistently improves SPL.

The result is the most legible curve in the paper: R2R success rises from 72% to 98% and RxR from 65% to 89%, while logged worker and improver costs decrease as the workspace evolves. Retained experience improves recurring-task performance and reduces cost without a single weight update. In detail, variants with and without ASENA-VLN converge to similar final success on R2R, while the policy keeps a clearer advantage on the longer RxR instructions; tool-equipped workspaces also maintain higher SPL on both benchmarks, so ASENA-VLN improves path efficiency even where final success converges.

11. Embodied question answering: higher accuracy on fewer steps

EQA requires an agent to explore an environment, gather visual evidence, and answer using as little motion as possible. Evaluation uses the released initial poses of HM-EQA and MT-HM3D under the Explore-EQA teleportation protocol: for a scene of area $A$, each episode has a nominal step budget of $3\sqrt{A}$; the paper reports answer accuracy and normalized exploration steps, the fraction of that nominal budget consumed:

$$B=3\sqrt{A},\qquad s=\frac{n_{\text{steps}}}{B} \tag{5}$$

Hand-designed reference configurations. The EQA agent composes navigation, visual inspection, object search, place memory, and answer-generation tools. Two hand-designed configurations serve as reference points: the quality configuration prioritizes collecting sufficient evidence to answer correctly and receives no explicit instruction to minimize exploration; the efficient configuration is prompted to treat exploration steps as a limited resource and gather evidence more selectively, with a target budget of 30% of the nominal allowance on HM-EQA, while both keep the same simulator hard limit. These provide manually designed operating points on the accuracy-efficiency frontier rather than a controlled ablation of a single component.

Evolution protocol and results. Fixed evolution pools contain 398 HM-EQA questions and 1,183 MT-HM3D questions from training scenes; in each pass sixteen workers sample 50 questions per benchmark, execute them with private workspaces, and reflect on their outcomes, while a separate improver aggregates their traces and edits to update the canonical workspace; unsampled questions are used for monitoring and their aggregate results are also given to the improver. With the workspace frozen, pass 17 achieves the highest HM-EQA accuracy at 81.2%, while pass 4 is best on MT-HM3D at 66.5%; at low reasoning effort, evolved workspaces use steps comparable to the hand-designed efficient configuration while achieving higher accuracy.

Method / configuration	HM-EQA acc.↑	HM-EQA steps↓	MT-HM3D acc.↑	MT-HM3D steps↓
Explore-EQA	58.4	.52	35.1	.64
GraphEQA	63.5	.20	45.6	.45
MemoryEQA (GPT-4o)	63.4	.40	55.1	.41
FAST-EQA	69.2	.65	50.5	.52
Qwen-RobotNav	76.7	.15	54.4	.19
ASENA base (low effort)	70.9	.15	64.8	.10
ASENA hand-designed (efficient)	72.2	.11	63.8	.11
ASENA hand-designed (quality)	78.8	.22	67.1	.24
ASENA self-evolved pass 17 (low)	74.8	.13	64.9	.08
ASENA self-evolved pass 17 (high)	81.2	.25	65.5	.22

The 81.2% HM-EQA accuracy is the best in the table at a cost of .25 normalized steps: more walking than the efficient configuration, but only slightly more than the quality configuration's .22 while being 2.4 accuracy points better. A paired ablation on 132 MT-HM3D questions shows ASENA-VLN reducing median normalized exploration steps by about 19% with little change in accuracy, the same mechanism as on R2R and RxR: learned navigation improves efficiency by reducing repeated agent interaction.

12. Real-world G1: four demonstrations without a pre-built map

The real-robot section presents four demonstrations, all operating without pre-built maps and combining navigation with search, user interaction, or object delivery. The three search missions take place in previously unseen environments with targets initially outside the robot's view: in the restroom mission the robot locates doors near the elevator lobby, uses its online map to identify a shorter route from the user, and returns with directions; in the pantry mission it locates the room, inspects a dispenser and a brewer, and reports two coffee machines; in the snack-comparison mission it inspects a reference snack, finds a vending machine, checks whether the same item is available, then returns to answer the user with a generated whole-body gesture. The fetch-and-handoff task (D) demonstrates humanoid mobile manipulation: after reaching the pantry the agent uses a head-camera observation to locate the can and ground a grasp, inspects an arm-motion plan, picks the can up with a YAM gripper, walks back while carrying it, and a separately programmed handoff uses inverse kinematics to position the can above the recipient's hand before releasing the gripper.

Figure 6 (paper Figure 5): real-world navigation, manipulation, and interaction on the G1. The four demonstrations pair online maps and recorded trajectories with observations and robot actions; blue and dashed orange traces show search and return; distances are estimated from drive odometry; D combines pickup, carrying, and handoff. Prompts and responses are condensed for display.

Hardware and runtime parameters are specified in detail in appendix A.5: the panoramic camera is natively 2880×1440 at 30 fps, configured to 6 Hz JPEG; the LiDAR delivers 200,000 points per second with a 360° horizontal and −7° to 52° vertical field of view, bridged by SLAM at 10 Hz; body state publishes at 20 Hz. The slow-walk controller bounds translation speed to 0.12–0.35 m/s and 0.25 m/s near the goal (within 0.5 m), with arrival tolerances of 0.10 m and 4°, move/turn command limits of 2.0 m and 180°, path chunk limits of 6.0 m and 16 points, a 25 s move-job timeout and 45 s path-job timeout, a 2.0 s velocity-command expiry, and a 3.0 s active-job heartbeat timeout. These runtime-adjustable limits form the second gate between "the agent writes programs" and "the robot moves safely". The paper also concedes that real-world tasks take 10–20 minutes each.

13. On-device inference: FP8 and a reduced vocabulary on Jetson Thor

To run the policy on the G1's Jetson AGX Thor, the paper applies post-training FP8 quantization with NVIDIA ModelOpt and builds inference engines with TensorRT-Edge-LLM; these measurements use an earlier checkpoint at step 8,400, separate from the benchmark policy in Table 1. Every FP8 variant quantizes the language model; variants labeled "BF16 vision" keep the 415M-parameter vision encoder in BF16; embedding and output projection stay unquantized in all variants. The decisive cut is the vocabulary: navigation-only engines need to predict motion tokens rather than general text, so the output vocabulary is reduced from 152,596 to 768 entries while retaining all 660 trajectory tokens, removing 388.7M parameters from the output projection evaluated at every decoding step.

Variant	Export size (GiB)	Visual (ms)	Prefill (ms)	Decode 32 steps (ms)	Total (ms)	Speedup
PyTorch BF16	9.01	167.3	363.4	2,275.5	2,806	1.00×
FP8, BF16 vision	5.63	73.3	128.7	847.4	1,050	2.67×
FP8, BF16 vision, reduced vocab	4.91	78.3	125.5	694.1	898	3.13×
FP8, reduced vocab	4.53	64.9	129.1	683.4	877	3.20×

Measurement protocol: batch size 1, the same eight-frame mixed-resolution history, and 32 greedy decoding steps; BF16 values are warm medians of five runs, TensorRT-Edge-LLM visual and prefill values are 10-run means, and decode is the median of five runs. FP8 with the reduced vocabulary lowers the sum of component latencies from 2,806 ms to 877 ms:

$$\frac{2806}{877}=3.20\times,\qquad \frac{683.4}{877}\approx 78\% \tag{6}$$

Decoding remains the largest cost at about 78% of the FP8 total; among BF16-vision variants, reducing the vocabulary cuts decoding from 847.4 ms to 694.1 ms (18.1%) while visual encoding and prefill stay nearly unchanged. Quantization agreement is checked on 256 held-out inputs: FP8 with BF16 vision reaches 91.41% first-waypoint agreement and 82.81% exact-trajectory agreement (pure FP8: 90.23% / 75.39%), so keeping the vision encoder in higher precision clearly helps output consistency. The paper stresses that these metrics measure output consistency rather than navigation success, which requires separate rollout evaluation; it also reports a measured BF16 end-to-end latency of 2,735 ms and 8.28 GiB of allocated memory after loading.

14. Limitations and outlook

The paper states its own boundaries plainly. The G1 demonstrations provide qualitative evidence of capability composition rather than a systematic measure of reliability across objects, layouts, and users; real-world tasks taking 10–20 minutes makes faster reasoning and execution a prerequisite for practical use. Future work includes evaluating retained skills on new tasks, measuring navigation and manipulation reliability over repeated real-world trials, and strengthening safeguards for dynamic environments; another direction is extending workspace evolution to updates of both the learned policy and the agent itself, i.e. putting the two weight families that are frozen today into the evolution loop.

Compressed into one engineering judgment: ASENA does not try to train a bigger end-to-end model that swallows every capability. It makes "the agent can write programs" and "the small policy can walk" two tools inside the same programmable interface, and lets experience accumulate as text and code. Frozen weights are not a limitation here; they are the precondition that lets the system keep evolving without losing control.

Golden line

Ten passes over the same tasks, success from 72% to 98%, and not one gradient update: what changed was the notes and skills in the workspace, what never changed was any model or controller weight. "Self-evolving" here is not rhetoric; it is a curve you can plot pass by pass.