Astranomy: An Empirical Study of GPT-6 Astra as a Native Robot Policy
NVIDIA studies GPT-6 Astra as a native robot policy: Agentic Policy tops RoboLab-120 at 74.8%, but each call averages 18.56 s and a successful episode costs 30.9M tokens ($36.40).
Robot foundation models are trained on robot data to become robot policies. GPT-6 Astra was trained on the internet. NVIDIA's Seattle Robotics Lab asked whether that difference actually matters: can a general-purpose multimodal model, with no robot-policy finetuning at all, close the loop on real manipulation tasks if you specify the control interface in the prompt? Their answer, published October 14, 2026 as the technical report Astranomy, is a qualified but striking yes. On the 120-task RoboLab benchmark, Astra reaches 74.8% success in its best configuration, beating every purpose-trained policy on the leaderboard, while averaging 18.56 seconds per inference call and burning up to 30.9 million tokens per successful episode.
A general-purpose model has the capability of operating as a native robot policy for general-purpose pick-and-place. The open question is no longer whether it can control the arm, but whether the latency and token bill justify it.
This post walks through the full report: the three ways the team wired Astra to a robot, the inference-time ablations that decide whether it works, the runtime economics that decide whether it is usable, and the failure modes that remain.
Key takeaways
- Capability is real. On RoboLab-120, Direct Policy scores 66.3% success, Agentic Policy 74.8%, and Orchestrated Policy 66.5%. The Agentic configuration beats the strongest leaderboard reference (Lakeside, 51.0%) by 23.8 percentage points, against purpose-trained VLAs and WAMs such as π0.5 (28.0%) and Cosmos3-Nano-Policy (36.8%).
- The harness matters as much as the model. The same model jumps from 66.3% to 74.8% purely by changing how it is called: a persistent Codex session with retained history instead of stateless API calls.
- Prompt geometry is the single biggest lever. For Direct Policy, adding gripper and camera calibration to the prompt lifts success from 19.3% to 72.7%. Without it the model cannot convert pixels into commanded motion.
- Memory beats pixels. In the agentic setting, retaining the model's own text history (plans, actions, reasoning) matters more than retaining old images; full history reaches 100% on a 23-task subset.
- Capability is not deployability. Inference consumes 94.9–97.8% of estimated episode walltime. A yogurt-placement task that executes in about 5.2 seconds of robot motion takes an estimated 3 minutes 57 seconds once Astra's thinking time is counted.
The question: trained robot policy, or prompted generalist?
Vision-language-action models and world action models adapt a pretrained backbone to control by finetuning on robot trajectories. That training fixes the action parameterization, the embodiment conditioning, and the temporal context: at inference time the interface is whatever the training data baked in. A general-purpose multimodal model has no such commitments. Its action representation, its notion of the robot's geometry, and its observation window are all things you write into the prompt and can change between runs.
That flexibility is the point of the study. Because Astra's context window is long enough to hold an entire episode, the authors can treat the control interface as an experimental variable rather than a design constant. One consequence is variable-length action chunking: for end-effector and joint-position control, Astra chooses how many actions to emit on each call within a prompted range, so the interval between observations adapts to the decision instead of being fixed by the model architecture.
Prior work had already hinted that this might work, evaluating Astra agentically on RoboDojo, as an embodied policy, and on navigation and humanoid control through pretrained whole-body controllers. Astranomy is the systematic version: it separates the model's own capability from the support its surrounding system provides, and it prices both.
Three ways to wire Astra to a robot
The report's central methodological move is to refuse the ambiguity in "the model controls the robot." Three configurations make the division of labor explicit.
| Configuration | Who produces actions | What the harness adds |
|---|---|---|
| Direct Policy | Astra alone, via stateless API calls | Nothing. Fixed observation-to-action mapping, no history retained between calls. Supports end-effector targets, joint targets, or joint-space B-splines with chunk length chosen within configured bounds. |
| Agentic Policy | Astra inside a persistent Codex session | Full interaction history, optional tools for image inspection and computation, persistent notes, and motion feedback after each action. One end-effector target per decision, held for 1–5 control steps. |
| Orchestrated Policy | Cosmos3-Nano-Policy proposes; Astra reviews | Astra acts as a monitor with bounded memory. It can accept a 15-step proposal from the finetuned world action model, edit it, or override with its own end-effector target. |
All three run closed-loop from three camera views (left shoulder, right shoulder, wrist) at 384×216 pixels plus proprioception and a language instruction, on the simulated DROID robot (a Franka Panda arm with a Robotiq 2F-85 gripper) at 15 Hz, with simulation paused during inference. Every configuration uses GPT-6 Astra at medium reasoning effort unless the model itself is the variable under test.
The benchmark: RoboLab-120
RoboLab is a high-fidelity simulation benchmark built to test generalist manipulation and robustness to language. It emphasizes multi-step tasks and deliberately varies instruction specificity while holding scene and goal fixed, so that sensitivity to wording can be separated from difficulty of physical execution. The full suite is 120 tasks; the ablations run on a 30-task subset spanning ten capability groups, from object selection and container placement through reorientation, shelf manipulation, counting and sequencing, to a long-horizon cleanup task with exclusion rules.
Two metrics are reported throughout. Success Rate (SR) counts episodes the simulator marks complete; Score additionally credits partial progress, so a run that places three of four objects is not scored the same as a run that knocked one over. Successful episodes score 100.
Headline results
On the full 120-task suite, the ranking is unambiguous. Agentic Policy leads at 74.8% SR / 83.6 score, Orchestrated Policy follows at 66.5% / 79.1%, and Direct Policy with end-effector actions at 66.3% / 76.7%. For reference, the strongest non-Astra entries on the leaderboard are Lakeside at 51.0%, FLUX 3 Action at 42.9%, and the purpose-trained Cosmos3-Nano-Policy at 36.8%; π0.5 lands at 28.0%.
Two details deserve attention. First, Direct Policy's score depends heavily on action representation: 66.3% with end-effector poses but only 50.2% with joint positions and 38.8% with B-splines, so "Astra as a direct policy" is not one number but a family of them. Second, the Agentic result is achieved by the same weights as the Direct result. Nothing was finetuned between the two; only the calling convention changed.
What the ablations reveal
Action representation: end-effector wins, but the harness closes the gap
On the 30-task subset, Direct Policy scores 72.7% with absolute end-effector poses, 48.7% with absolute joint positions, and 37.3% with joint-space cubic B-splines. Absolute representations beat delta and relative variants in every family. The B-spline condition is worth a closer look because it is the most ambitious interface: the policy emits six eight-dimensional coefficient vectors, each holding seven absolute joint positions plus a gripper command, and the client builds a clamped cubic spline through them,
$$C(u)=\sum_{i=0}^{6} N_{i,3}(u)\,P_i,\qquad u\in[0,1],$$
with the current state prepended as $P_0=[q_{\text{current}};\ \mathbb{1}(g_{\text{current}}\geq 0.5)]$ over the clamped knot vector $(0,0,0,0,\tfrac14,\tfrac12,\tfrac34,1,1,1,1)$, then sampled at $C(k/H)$ for an action chunk of length $H$. Astra can produce these natively, but it does not produce them well enough yet to compete with plain end-effector targets.
Under the Agentic harness the picture changes: end-effector 86.0% versus joint positions 84.0%, a gap of only 2.0 points. When the model can re-observe, re-plan, and correct after every short segment, the choice of action space matters far less. The authors also tested action horizon: allowing chunks of 16–32 actions cut model calls by 62% with similar completion time, but cost 6 points of success, because longer open-loop segments cannot correct mid-flight. The default dynamic 3–15 range is the sensible middle.
Geometry in the prompt: the largest single effect in the report
Direct Policy with only a minimal prompt, 19.3% success. Add gripper geometry (flange-to-finger-pad offsets, jaw orientation, a lookup from gripper closure to pad midpoint) and camera geometry (extrinsics, normalized intrinsics, wrist-camera tilt) and the same model reaches 72.7%. The intermediate steps are 35.3% for gripper-only and 62.7% for camera-only. The effect is visible in the motion itself: with the base prompt the arm hovers and re-aligns repeatedly over the yogurt cup; with full geometry it approaches, grasps, and places in one clean sequence.
In the Agentic setting the same geometry text moves success only from 83.3% to 86.0%, but removing it costs 14.7% more control steps. The authors' hypothesis is that a persistent agent can infer geometry from its own observation-action history and correct alignment incrementally across short segments, so the prompt stops being the only source of calibration. The practical reading: if you run a generalist statelessly, the calibration text is not optional decoration, it is the interface.
Context: memory matters more than pixels
For Direct Policy, giving only the current observation scores 49.3%; adding two earlier observations at t−14 and t−7 lifts it to the 72.7% default. Making the window longer (t−29) or denser (five frames) adds nothing, so roughly one to two seconds of visual history is where the value saturates.
The agentic context ablation is more striking. Five conditions separate text history (the model's own prior messages, plans, action calls and reasoning records) from visual history. Full text plus full visual history reaches 100% on a 23-task subset. Full text with only current images still reaches 87.0%, while full visual history with no text history collapses to 56.5%. Retaining everything the model previously said and did is worth more than retaining everything it previously saw. Selecting only the two most recent assistant turns (73.9%) also underperforms full text history, suggesting the value lies in the complete record of what was tried, not in a rolling summary.
Task specification: wording sensitivity is a Direct-Policy problem
Replacing a precise instruction ("Put the small red yogurt in the red bowl") with a vague one ("Put it away") costs Direct Policy 8.7 points (72.7% → 64.0%) but costs the Agentic Policy almost nothing (86.0% → 85.3%). A persistent agent can resolve ambiguity from context and from what it has already done; a stateless caller cannot.
The report also tests in-context learning from video: frames sampled at 1 Hz from a previous successful rollout of the same task, all three views, supplied instead of a language instruction. Video-conditioned execution reaches 87.9% versus 90.0% for language on the matched subset. Demonstration frames are a usable specification, but on this benchmark a sentence is still the better interface.
Model choice: the capability is not generic
Under the identical Direct Policy configuration, GPT-6 Astra scores 72.7%, GPT-6 Sol 33.3%, Claude Opus 5.5 25.3%, and GPT-6 Luna 2.0%. Native closed-loop control is a capability of the frontier model in this family, not an emergent property of any large multimodal model. Raising reasoning effort from medium to xhigh changed success little while increasing latency and token cost, so the default stays at medium.
Agentic scaffolding: tools and planning are optional, history is not
The Agentic harness offers optional shell tools: Python execution, image crops, geometric calculations, persistent notes. When enabled, every one of the 150 episodes used them, averaging 19.7 tool calls per episode, including stereo triangulation, gripper-pose arithmetic, inverse kinematics and enlarged crops. Pooled success moved from 86.0% to 88.0%. The audit's honest conclusion is that Astra often reaches the same decision with or without the tools; they are available competence, not measured advantage.
Explicit planning instructions show the same pattern. Asking the model to decompose the task into subgoals and track progress yields 86.0%; removing the instruction yields 90.0%; actively discouraging planning yields 81.0%. Rollout transcripts show progress tracking appearing even when planning is discouraged, because the retained conversation history already contains the record of completed subgoals. Motion-error feedback is similar: provided by default, it appears explicitly in 51% of episodes but in only 1.14% of next-action explanations, and removing it moves success from 85.9% to 85.2%. The agent reconstructs tracking error from its own action history and fresh proprioception.
Orchestrated Policy: what a frontier model catches in a learned policy
When Astra monitors Cosmos3-Nano-Policy, it intervenes on 14.5% of all simulation steps. Grouping consecutive intervention decisions into 4,254 blocks and classifying them with a VLM, grasping problems account for 64.4% of interventions and wrong-target or planning mistakes for 20.2%; placement, manipulation and transport failures make up most of the rest, with 0.3% unclear. Ten tasks required no intervention at all. The reading is that a finetuned world action model is competent at reaching and transporting but remains weakest exactly where closed-loop grasp correction is needed, and that is where a reasoning monitor adds value. Notably, the orchestrated combination (66.5%) did not beat Astra generating actions natively in the agentic harness (74.8%), so delegating to the specialist is not yet a free win.
The bill: latency and tokens
Capability is the headline; the runtime table is the fine print. Across successful RoboLab-120 episodes, Direct Policy with end-effector actions averages 18.56 seconds per inference call, and inference accounts for 97.0% of estimated episode walltime. Agentic Policy averages 13.37 s per call (97.8% of walltime); Orchestrated splits into 11.50 s for Astra decisions and 2.74 s for the WAM proposals (94.9% combined). Purpose-trained references are two to three orders of magnitude faster per chunk: π0.5 at 46.96 ms, Cosmos3-Nano at 713.76 ms, FLUX 3 Action at 397.74 ms on H200 in BF16.
| Configuration | Latency per call | Inference share of episode | Tokens / cost per successful episode |
|---|---|---|---|
| Direct Policy · EEF | 18.56 s | 97.0% | 160k / $1.96 |
| Direct Policy · jointpos | 20.19 s | 97.4% | — |
| Direct Policy · B-spline | 21.68 s | 97.0% | — |
| Agentic Policy | 13.37 s | 97.8% | 30.9M / $36.40 |
| Orchestrated Policy | 11.50 / 2.74 s | 94.9% | 11.0M / $17.72 |
| FLUX 3 Action (WAM) | 397.74 ms | 16.1% | n/a |
| Cosmos3-Nano-Policy (WAM) | 713.76 ms | 25.6% | n/a |
| π0.5 (VLA) | 46.96 ms | 4.6% | n/a |
The token economics are the sharper edge. Direct Policy costs about 160k tokens and $1.96 per successful episode. Agentic Policy, which buys the extra 8.5 points of success, consumes 30.9 million tokens and $36.40 per successful episode, roughly 190× the token count. Orchestrated sits between at 11.0M tokens and $17.72. The yogurt task video makes the latency concrete: about 5.2 seconds of actual robot motion inside an estimated 3 minutes 57 seconds of episode, the rest spent waiting for the next action chunk. Any application where the world moves while the model thinks is out of reach at these numbers; simulation paused during inference is what makes the results possible at all.
How and when Astra fails
The failures are informative rather than exotic. The recurring limitation across RoboLab-120 is converting an attempted manipulation into a completed grasp, transfer, or placement: the gripper keeps adjusting around an object without securing it, or an object is displaced without ever reaching its destination. Online motion correction does not guarantee physical progress. In the agentic setting the persistent failure list is incorrect target selection, unstable or obstructive grasps that trap the gripper, incomplete manipulation sequences, objects lost in transport, and recovery attempts that repeat ineffective motions or never retrieve an object that fell off the table. Assistance improves reliability on average but does not consistently resolve grasp error, and the benefit varies task by task.
There is also a deployment-shaped failure mode that no prompt fixes: at 18.56 s per call, a task that requires reacting to a fast-changing scene is simply not addressable. The report is explicit that these results establish capability within a paused simulator, not real-time competence.
Limitations worth holding in mind
- Five episodes per task. Inference cost caps the trial count, so run-to-run variability is not characterized and the smaller ablation gaps are not established as statistically significant. Treat the ablations as evidence of which design choices deserve study, not as settled effect sizes.
- Paused simulation, pick-and-place only. No contact-rich insertion or assembly, no long-horizon mobile manipulation, no dynamic tasks, and nothing that moves while the model thinks.
- Interface-level visibility. The models are evaluated through their public interfaces; internal serving-side processing cannot be separated from model capability, so the ablations characterize the deployed system's response to interface changes.
Bottom line
Astranomy reframes the robot-policy question from "can a generalist do it" to "under what interface, and at what price." The interface answers are concrete and transferable: absolute end-effector targets, a second or two of visual history, explicit gripper and camera calibration in the prompt, and, above all, let the model remember what it already said and did. The price answers are equally concrete: 97% of the walltime is thinking, and the configuration that performs best costs thirty-six dollars a success. For offline, high-value, low-dynamism manipulation, a prompted frontier model is now a legitimate policy option and, on this benchmark, the best one measured. For anything that moves in real time, the purpose-trained models remain the only viable controllers, and the interesting open question the report leaves behind is which decisions are worth delegating between the two.
Source: Astranomy: An empirical study of GPT-6 Astra as a native robot policy, Yang, Khetan & Zhang, NVIDIA Seattle Robotics Lab, October 14, 2026. Figures and videos reproduced from the original report.
Source:NVIDIA Research (Seattle Robotics Lab)https://research.nvidia.com/labs/srl/projects/astranomy/