Skip to content
←Back to the radar

SOTA · language

Reasoning

General reasoning, scientific QA, long-chain inference and planning

Reasoning is the only part of language capability that maps directly onto "how far from AGI": it does not ask whether a model talks well but whether it carries multi-step inference correctly on structures it has not seen. Its relation to AGI has three layers. One, it is upstream of every agent behaviour: planning, tool selection and error recovery are all special cases of reasoning, and when the chain breaks a long task necessarily fails, which is why the ceiling of `agents` and `harness` traces back here. Two, it is one of the few capabilities checked automatically: mathematics and code supply objective right-and-wrong signals, so "got stronger" can be measured rather than merely claimed. Three, it exposes the real limit of the current paradigm: scores rise while generalisation to new rules stays unstable. The ladder is therefore ordered by third-party composite readings, not vendor-reported single points: on the synced Artificial Analysis intelligence index (2026-09-22) the leader is Claude Opus 5.5 (max 57.62, xhigh 55.99, high 53.58) with GPT-6 Astra (max) at 52.67 behind, while the net-improvement leader on the Arena Agent board is Claude Fable 5.1 (13.71), which measures how far a real task gets pushed forward and is closer to production usefulness than a question-answering score. All three are now standalone asset pages, each carrying a close reading of positioning, capability surfaces, water level and limits, and stating the verification not done (capability reproduction, long-task comparison on an in-house harness); grade A rests on official documentation and third-party boards agreeing, not on any reproduction. Selection: Opus 5.5 for most workloads at $4/$20, Astra for terminal engineering on Terminal-Bench 58.18%, Fable 5.1 for real tasks at 2.5x the price for 4.3 index points.

RULERGPQA, MMLU-Pro, BBH, ARC-AGI

The ladder

7
ReasoningDevelopmentTopA

Claude Opus 5.5: the long-running agentic tier at $4/$20 with 1M context and adaptive thinking

The fixed version Anthropic positions for long-running agentic coding and knowledge work (claude-opus-5-5, retirement no earlier than 2027-09-22): 1M-token context, 128K-token single output, adaptive thinking defaulting to medium, at $4/$20 per million tokens, the cheapest of the three top rows tracked here. On the Artificial Analysis intelligence index (2026-09-22) the top three rows are all its thinking tiers (max 57.62, xhigh 55.99, high 53.58), while the default medium tier sits #8 at 51.24, with sibling Fable 5.1 (53.35) and GPT-6 Astra (52.67) in between. Two discounts: those rows carry the with fallback qualifier while Astra reads (max), so the protocols differ, and the 53.94% at #8 on Terminal-Bench belongs to the previous-generation Opus 5. Closed, API only; adaptive thinking is a black box, so budget on p90 rather than the mean; 1M context is not 1M of effective attention, so whole-repo input still needs retrieval. Use it for most workloads and move to Fable 5.1 only when the highest tier is not enough: 2.5x the price for 4.3 index points. Graded A (confirmed); not independently benchmarked.

57.62Artificial Analysis Intelligence IndexConfirmed · 2026-09
ProductionAnthropicSite
Claude Opus 5.5: the long-running agentic tier at $4/$20 with 1M context and adaptive thinking
ReasoningDevelopmentTopA

Claude Fable 5.1: the long-horizon agentic tier that tops the ladder on a different ruler

The fixed version Anthropic positions for demanding reasoning and long-horizon agentic work (claude-fable-5-1, retirement no earlier than 2027-09-01): default thinking effort high at $10/$50 per million tokens, 2.5x sibling Opus 5.5, and default-high is itself a default cost behaviour. Its standing only holds once you switch rulers: #1 on net_improvement in the Arena Agent board (13.71, confirmed_success 19.83, praise 31.83) ahead of GPT-6 Astra (11.54/17.70/32.79), while Opus 5.5 misses the top eight; #2 on Terminal-Bench 4.0 at 57.88% (n=330), 0.3 points behind Astra at 58.18% but with pass@5 of 0.7879 against 0.7121, lower single-shot and more robust over five attempts; #4 on the intelligence index at 53.35 (max), below Opus 5.5. The picture is consistent: not first on composite intelligence, first on pushing a real task forward. A separate praise column means the board carries a human or judge component and is not an objective benchmark. Choose by whether the workload is hard from reasoning or from long-horizon consistency. Closed, API only. Graded A (confirmed); no long-task comparison on a standard harness.

13.71Arena Agent net improvementConfirmed · 2026-09
ProductAnthropicSite
Claude Fable 5.1: the long-horizon agentic tier that tops the ladder on a different ruler
ReasoningDevelopmentTopA

GPT-6 Astra: first on terminal engineering tasks, third tier on composite intelligence

The OpenAI flagship tier, officially positioned for hardest end-to-end work (gpt-6-astra): 1.05M-token context, 128K-token single output, knowledge cutoff 2026-04-30, with Functions, Web search, File search and Computer use built in, so search, retrieval and interface control ship with the model instead of an outer agent framework. Reasoning effort runs low to max in five steps while price stays fixed at $10/$50, so the cost lever is token consumption; siblings Sol ($2/$10) and Luna ($0.1/$0.5) make a 100x spread, so load splitting can stay inside one vendor. Readings (2026-09-22): #1 on Terminal-Bench 4.0 at 58.18% (n_trials=330, pass@5 0.7121), the protocol closest to a coding agent's daily life; #2 on Arena Agent with net_improvement 11.54 and confirmed_success 17.70, both below leader Fable 5.1, but the field's highest praise at 32.79; #6 and #7 on the intelligence index (max 52.67). The cutoff is nearly five months older than this page, so version numbers and API changes must go through Web search; computer-use reliability appears on no board; closed, API only. Graded A (confirmed); not independently benchmarked.

58.18%Terminal-Bench accuracyConfirmed · 2026-09
ProductOpenAISite
GPT-6 Astra: first on terminal engineering tasks, third tier on composite intelligence
LLMDevelopmentTopC

Jev: TypeSafe's first System One Model types decisions and prices them two orders of magnitude lower

TypeSafe's first System One Model: outputs are type-safe structured values under three question primitives (Noul/Choice/Score) rather than strings, trained with RLCD for calibrated decision probabilities. The official 13-question parallel experiment makes one synthesized call ~12.2x cheaper and ~10.0x faster than 13 separate calls; four-workflow mean accuracy 67.8% at /tmp/run_ingest2.sh.0004 and 0.4s per instance - opus 5 / sol tier accuracy at almost two orders of magnitude less cost; input $0.042/M tokens, output free. Ships an OpenJev local reproduction path and an eight-item caveats list.

67.8%Mean accuracy across four workflows (vendor eval)Vendor Claim · 2026-09
ProductTypeSafeSiteRepo
Jev: TypeSafe's first System One Model types decisions and prices them two orders of magnitude lower
LLMDevelopmentTopA

MiMo-V2.6: Live RL as an auditable ledger of 30 steps, 750k trajectories, and dollars

Xiaomi's open-source MiMo-V2.6 Pro/Flash: 30 steps of Live RL, roughly 750k trajectories, public cost of $850k/$2.62M, and 7k+ RL environments released in the same batch; tops the open-weights camp on the Artificial Analysis intelligence index at 46 with +17/+14 points out-of-sample on DeepSWE v1.1; four demo lines (Vibe World, CUA, research, content creation) all open. Self-improvement becomes an auditable ledger - steps, trajectories, and dollars on the table for third-party recomputation.

46Artificial Analysis Intelligence Index (top open-weight)Confirmed · 2026-09
ProductXiaomiSiteRepo
MiMo-V2.6: Live RL as an auditable ledger of 30 steps, 750k trajectories, and dollars
LLMDevelopmentC

Laya: a 421M non-autoregressive System 1 decision model at 33 ms per forward pass

Convai Innovations' open-source non-autoregressive System 1 decision model: 421M parameters, ModernBERT backbone with [MASK] option extraction, one 33 ms forward pass per calibrated decision, Apache-2.0; trained with RLCD strictly proper scoring rules for honest probabilities; the model card ships a real benchmark against Jev plus an honest limitations list. Positioned for edge and high-concurrency small decisions (game NPCs, dialogue policy, request routing), not long reasoning or open generation.

33 msSingle forward-pass latency (421M, vendor)Vendor Claim · 2026-09
ResearchConvai InnovationsSiteRepo
Laya: a 421M non-autoregressive System 1 decision model at 33 ms per forward pass
AI Coding

OpenResearch: A Local-First Workspace That Turns Coding Agents into Research Agents

OpenResearch (orx) is the local-first research workspace open-sourced by alphaXiv: a single Rust binary that turns Claude Code, Codex, OpenCode, Cursor, or Google Antigravity into research agents that review literature, form hypotheses, run experiments, and produce artifacts. It organizes work as a git-native experiment tree under a fixed run contract, drives experiments through a repair/refill/promote/stop autoresearch loop, and treats run logs and manifests as the only evidence. Written from a close read of the v0.2.13 source and a hands-on run of the bundled nanochat demo.

6.2kGitHub stars
OpenResearch: A Local-First Workspace That Turns Coding Agents into Research Agents

Latest demos

19

Papers & deep reads

29
DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses2026-10-16DiscoveryRSI: Recursively Learning to Discover Valuable HypothesesScientific discovery requires developing worthwhile hypotheses into practical investigations, yet learning from this evolving research process remains challenging. We introduce an operational definition of valuable and testable research hypotheses: evidence-grounded claims about consequential uncertainties, coupled with feasible investigations, explicit decision criteria, and alternative explanations. Our DiscoveryRSI framework develops hypothesis trees through multi-agent evolutionary search, compiles selected investigations into executable Harbor tasks, and recursively learns through three mechanisms: hypothesis credit optimization attributes scientific assessments to documented research contributions; harness replay identifies productive strategies and recurring failures to refine research workflows; and rediscovery applies accepted improvements to new investigations, generating further experience for subsequent learning. We also introduce DFM-Bench, which provides comprehensive assessment of hypothesis quality and cumulative research progress at the levels of hypotheses, revisions, development paths and research runs. Across four life-science domains, automated evaluation shows that hypothesis quality rises from 4.86 for the base model to 5.39 after two rounds of self-improvement, with the large gains in falsifiability and data resolvability. We further validate hypothesis quality through expert review, comparing our DiscoveryRSI-27B with other research agents. Preliminary human assessments show that DiscoveryRSI-27B scores higher than GPT-6 Luna on hypothesis importance and whether claims are plausible and evidence-bounded. DiscoveryRSI-27B achieved a 6.3% rate of hypothesis correspondence with research published after the model cutoff, largely outperform GPT-6 with 2.5% and Claude Opus 5 with 2.3%, while delivering greater human-rated improvement along research paths.
HIL-UMI: Bringing Human-in-the-Loop Post-Training of Vision-Language-Action Models to Universal Manipulation Interface2026-09-17HIL-UMI: Bringing Human-in-the-Loop Post-Training of Vision-Language-Action Models to Universal Manipulation InterfaceLarge-scale vision-language-action (VLA) models provide powerful priors for robot manipulation, yet adapting them to a specific deployment remains challenging. Supervised fine-tuning (SFT) on task-specific demonstrations provides a step toward deployment, but faces two persistent limitations: static data provide limited coverage of out-of-distribution states, and standard imitation objectives do not distinguish progressing behavior from less useful data. Interactive post-training can address these limitations, but typically requires repeated policy execution and human intervention on a physical robot. We introduce HIL-UMI, a policy-guided Universal Manipulation Interface (UMI) framework for robot-free human-in-the-loop VLA post-training. During handheld UMI demonstrations, HIL-UMI queries the current policy on the same observation stream without executing its predictions. The Energy Score compares the human action trajectory with policy inference and triggers collection when their discrepancy indicates an out-of-distribution region. In a separate feedback loop, low online advantage predictions identify essential segments for refining a progress-based advantage estimator. The updated estimator then guides advantage-conditioned behavioral cloning using a balanced mixture of base demonstrations and new policy data. This design preserves the iterative and policy-aware nature of human-in-the-loop learning while decoupling data collection from robot deployment. Experiments on four real-world tasks spanning long-horizon and precise manipulation show that HIL-UMI achieves consistent improvement over SFT and benefits from both targeted collection and advantage refinement. Moreover, HIL-UMI outperforms HG-DAgger on Clean Up Table with lower per-frame collection time, suggesting a scalable path for VLA post-training across operators and locations.
Dream-RSI: Recursive Self-Improvement through Evolving Worlds2026-09-14Dream-RSI: Recursive Self-Improvement through Evolving WorldsRecursive self-improvement is becoming increasingly vital for autonomous AI agents, where progress hinges on discovering high-value solutions across complex domains. The driver of this process is effective exploration, however, managing and improving exploration strategies remains a major bottleneck. Current systems face a fundamental dilemma: fixed strategies fail to adapt as search spaces scale, while online policy optimization requires navigating vast meta-search spaces under delayed and expensive feedback over long-horizon rollouts. We introduce \textsc{Dream-RSI}, a framework for scalable and recursively self-improving exploration. A lightweight orchestration layer makes exploration explicit and programmable while leaving the underlying coding agent unchanged. Our key insight is that accumulated discovery history can serve as a replay simulator over the realized search space. By performing dreaming in the replay simulator constructed from historical discovery trees, \textsc{Dream-RSI} secures immediate, low-cost off-policy feedback to evaluate and refine exploration policies without invoking repetitive, expensive online evaluations. The improved policy is subsequently redeployed online to drive further discovery, continuously expanding the simulator pool in a self-improving loop. Across algorithm engineering, mathematical optimization, and GPU kernel engineering, \textsc{Dream-RSI} achieves competitive or improved discovery quality while substantially reducing discovery cost in several settings.
RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs2026-09-11RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any InputsOpen-vocabulary detection accepts any class list at inference, and promptable segmentation returns regions without class names: the taxonomy has left the model and become an input. Relation prediction has not. Scene-graph models are still trained and evaluated on the 50 or 56 predicates of one annotation style, their relation head conditioned on object labels and so tied to one detector. Three obstacles explain this, none primarily modelling: no relation corpus is both free-text and verified, a label-conditioned architecture cannot accept a vocabulary it was not trained on, and the standard metric rewards agreement with the training corpus, so a larger vocabulary scores as a regression. We present RelateAnything, a 53M-parameter model taking an image and regions from any source and returning scored relations over a predicate vocabulary supplied at inference as strings. Object labels are never an input, so the region source can change without retraining, and the vocabulary is a bank of text embeddings, not a learned classifier. It runs at 20 ms/frame. Training over 19,103 predicates requires positive-unlabeled supervision and a text encoder that separates antonyms, which contrastive encoders embed at cosine 0.95. To supply the supervision we build RA-4M, 474k images and 4.3M relations over 10,102 free-text predicates, generated against numbered box markers and geometrically verified. To measure it we build OV-SGG-Bench, six axes scored across datasets that the priors standard recall rewards cannot satisfy. On three cross-dataset benchmarks and a fourth zero-shot, RelateAnything has 2.3-3.5x the mean recall of the strongest open-vocabulary method of comparable scale, margins that survive a real detector, and leads a 3B-VLM scene-graph model on both metrics at under 2% of its parameters. In-domain measurement overstates transfer gains ~5x. Model, corpus and benchmark are public.

Open source

3

Boundaries & failure modes

Almost none of the numbers in this domain can be taken at face value, so the boundaries need detail. One, contamination: GPQA, MMLU-Pro and BBH have been public for years and training data almost certainly contains same-source items, so a high score is not strong reasoning; ARC-AGI matters precisely because it rotates in new tasks each round. Any vendor-reported reasoning score is therefore defaulted to vendor-claim, and confirmed is given only to readings from the synced independent protocols (the Artificial Analysis intelligence index, Arena Agent, Terminal-Bench). Two, "effort tiers" make numbers non-comparable: the same model can differ by four index points across high / xhigh / max (Opus 5.5 measured at 53.58 / 55.99 / 57.62), while launch materials usually quote only the top tier, and comparing your max against someone else's default is the most common misreading in this space. Three, fallback is folded into the index: rows whose board name says "with fallback" allow the model to give up and switch strategy, so they are not a pure reasoning reading. Four, the real magnitude of cost and latency: a single max-effort long inference can burn several times the tokens of the default tier, and on Terminal-Bench the gap between GPT-6 Astra at 58.18% (pass@5 0.7121) and Fable 5.1 at 57.88% (pass@5 0.7879) translates in a budget into "how many retries before success"; one row on the SWE-bench board carries an explicit price tag - mini-SWE-agent + Claude 4.5 Opus resolves 76.8% at a cost of $376.95. Five, error accumulation over long chains: at 95% per step, twenty steps leave 36%, and that is where the current ceiling on long-horizon agent completion comes from, not from models being "not smart enough".

As an Amazon Associate, we earn from qualifying purchases.