LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks

Introduction

A long-horizon task is ultimately solved as a series of subtasks, yet the right decomposition cannot be fixed in advance: the next subtask depends on what has actually happened in the environment so far. Most existing agent harnesses nevertheless keep a whole long-horizon task in one continuously growing session, where step-level execution competes with long-term coordination, and progress is judged inside the very session that produced it.

LongHorizon-Harness, released by AMAP-ML (the DreamX team), takes the opposite stance: it keeps the task state as an explicit record outside execution, admits only independently verified facts into that record, and derives each next subtask from it. The result is a Manage–Execute–Audit (MEA) loop of independently audited state transitions — and, with the exact same backbone model and execution backend, large gains on every benchmark tested: WeaveBench PassRate jumps from 51.8% to 80.7%, OSWorld 2.0 binary completion improves by 3.0× (2.8% → 8.3%), and Terminal-Bench 2.1 success rises from 69.7% to 77.2%.

Why Single-Session Execution Fails

The authors identify three failure modes that all trace back to the same architectural choice — coupling execution, state, and self-assessment in one growing context:

Compounding errors

An early mistake distorts every choice made after it, and the agent drifts away from the goal it started with.

Context rot

As the history grows, what matters gets harder to retrieve, and performance falls off sharply once context use crosses a threshold.

Task-state loss

There is no accurate record of what is done, what was produced, and what the environment actually contains, so progress cannot be resumed.

Once an unverified premise enters the record, every later decision inherits it. Existing harnesses never separate progress assessment from execution: a subagent reports a summary of its own work, and review is an optional action rather than a mandatory gate. Unverified premises accumulate and lead to repeated attempts, goal drift, or premature termination. An audit gate removes that failure at its root.

Core Idea: Keep the Goal Fixed, Ground Every Decomposition in Audited Facts

LongHorizon-Harness therefore replaces the single growing session with a loop over three separated roles, so the task is decomposed dynamically and every decision is grounded in independently verified facts rather than self-reports. The design rests on four pillars:

• P1 · Dynamic decomposition, fixed goal. Every round, the manager reads what has been verified as done, which constraints remain active, and what is still missing, then derives the next subtask with its target, dependencies, boundaries, and acceptance criteria.
• P2 · Audit-grounded progress. Only a read-only auditor can update the record. It inspects the real environment after each round and never sees the executor’s reasoning or self-assessment, so a fact is completed only when an audit report cites it.
• P3 · Round-local execution. A fresh-context executor performs the single current subtask and discards its context afterwards. Only compact audit reports cross rounds, so dense step-level observations never reach long-term coordination.
• P4 · Reset without amnesia. An execution error is either exposed by the next audit or never enters the record, so it stays confined to its round. The audit report separates what remains trustworthy from what must be repaired.

Method: Three Structurally Isolated Roles, One Audited State Machine

LongHorizon-Harness preserves the native agent loop of existing systems. A lightweight AgentAdapter lets Claude Code, Codex CLI, OpenClaw and Hermes Agent serve as interchangeable backends for all three roles, spanning models such as Claude Opus, GPT and Qwen — a different model per role, assigned by configuration alone. Each role has its own budget: 1800 s per round for the executor, 300 s for the manager and auditor, under a cap of 25 MEA rounds per task.

Manager (M)

Keeps the task state: every requirement, artifact and environment fact, each marked completed, pending, blocked or untrusted, together with the audit evidence behind that status. A record becomes completed only when an audit report supports it. Each round it reads that state and decides whether to run another subtask, finish, declare the task blocked, or ask the user.

Executor (E)

The only module allowed to change the environment. It receives one subtask and the evidence that subtask depends on, and starts from a fresh context every time; its raw trajectory and reasoning are discarded afterwards. Whether it works through the GUI or the command line follows the change being made, not what tools it happens to hold.

Auditor (A)

Checks the result independently, from a context that never sees the executor’s trajectory or reasoning. It may follow the executor’s report to find the outputs, but it decides completion by inspecting the environment itself against the acceptance criteria, and records what it verified, what is still unresolved, and the evidence for each conclusion.

Read-only integrity

Auditing is restricted to read-only interaction, so verification cannot alter the result under inspection. The harness monitors task-relevant artifacts and workspace contents while an audit runs; if the auditor modifies protected state, its report is marked an integrity violation and can no longer support a completed record.

One Round of the MEA Loop

Each round applies three operators in sequence: state transition (manage), state-changing action (execute), and state capture (audit). The auditor’s report from round i feeds the manager’s decision in round i+1, which in turn shapes the executor’s context in that same round. Concretely, one round has six steps:

1. Manager reads the record. It takes the original task, the current task state, and every audit report gathered so far.
2. Apply the verified findings. Confirmed changes add or update requirement, artifact and fact records; anything unresolved stays pending, blocked or untrusted. Nothing is marked completed without audit evidence.
3. Emit one contract. The manager picks an unresolved objective reachable from the current state and bounds it: goal, acceptance criteria, boundary constraints, and the prior audit evidence relevant to doing and checking it.
4. Executor acts, then forgets. It starts from a fresh context holding only what this round supplies, changes the environment, and reports what it did. Its raw trajectory is then discarded.
5. Auditor captures the state. It inspects the environment through read-only tools against the contract’s criteria. The executor’s report may guide where to look, but cannot establish completion.
6. Continue, or stop. The manager returns done when the audited state satisfies the task, blocked when nothing feasible advances it, ask when it needs user input, otherwise execute with the next contract — up to 25 rounds.

Benchmarks: Three Long-Horizon Axes of Difficulty

The harness is evaluated on WeaveBench, OSWorld 2.0, and Terminal-Bench 2.1, covering cross-interface coordination, long-horizon state management under realistic professional complexity, and pure CLI competence. Runs use Qwen 3.7-Plus as the primary backbone and Claude Opus 4.7 for the backbone-generality study.

	WeaveBench	OSWorld 2.0	Terminal-Bench 2.1
Scale	114 tasks across 8 domains	108 tasks, median 1.6 h for a human	Hard CLI tasks, 3 runs each
Interface	GUI and CLI in one trajectory	Desktop GUI, 1920×1080	CLI only — no visual perception
Scoring	Trajectory-aware agentic judge; PassRate counts tasks scoring ≥ 0.8	Native env.evaluate(); binary and partial	Per-task pass/fail, averaged over runs
Environment	Containerized VM, frozen snapshot, restricted network	Official osworld-v2-2026.06.24 Docker VM	Harbor on the Docker backend
What it tests	Whether evidence gathered in one interface survives into another	Whether task state survives an hour-long professional workflow	The state-management layer alone, with GUI routing removed

Results: Consistent Gains Across Benchmarks and Backbones

With the same backbone and the same execution backend, LongHorizon-Harness lifts WeaveBench PassRate from 51.8% to 80.7%, OSWorld 2.0 binary completion by 3.0×, and Terminal-Bench 2.1 success from 69.7% to 77.2%. The gains carry over from Qwen 3.7-Plus to Claude Opus 4.7.

Table 1 · WeaveBench results (114 tasks). PR denotes PassRate (%), Overall the mean per-task score; DSK/DOC/GAM/WEB/DAV/OPS/SPA/DES are the eight domains.

Agent	Harness	PR ↑	Overall ↑	DSK	DOC	GAM	WEB	DAV	OPS	SPA	DES
Claude Opus 4.7	Claude Code	41.2	0.532	55.6	47.1	23.5	53.3	23.1	50.0	33.3	40.0
Claude Opus 4.7	OpenClaw	35.1	0.482	55.6	29.4	23.5	66.7	15.4	41.7	16.7	20.0
Claude Opus 4.7	Hermes Agent	28.1	0.516	33.3	47.1	11.8	26.7	30.8	50.0	8.3	10.0
Claude Opus 4.7	Codex CLI	13.2	0.378	16.7	11.8	11.8	6.7	7.7	25.0	16.7	10.0
GPT-5.5	Codex CLI	35.1	0.499	38.9	29.4	23.5	53.3	15.4	50.0	58.3	10.0
GPT-5.5	OpenClaw	33.3	0.466	38.9	35.3	35.3	21.4	23.1	38.5	33.3	40.0
GPT-5.5	Hermes Agent	31.6	0.466	55.6	29.4	35.3	40.0	7.7	25.0	25.0	20.0
GPT-5.5	Claude Code	14.9	0.299	33.3	11.8	11.8	0.0	15.4	16.7	25.0	0.0
GPT-5.4	OpenClaw	22.8	0.465	55.6	35.3	5.9	0.0	23.1	23.1	8.3	20.0
GPT-5.3-codex	OpenClaw	18.4	0.456	33.3	23.5	29.4	0.0	7.7	16.7	8.3	20.0
GPT-5.2-codex	OpenClaw	6.1	0.321	5.6	11.8	0.0	0.0	15.4	16.7	0.0	0.0
GPT-5.1-codex	OpenClaw	1.8	0.226	0.0	5.9	0.0	0.0	7.7	0.0	0.0	0.0
Gemini 3.1 pro	OpenClaw	1.8	0.223	0.0	0.0	0.0	0.0	0.0	8.3	8.3	0.0
Qwen3.5-397B-A17B	OpenClaw	0.9	0.318	0.0	0.0	0.0	0.0	0.0	8.3	0.0	0.0
Qwen3-VL-8B-Think	OpenClaw	0.9	0.092	0.0	0.0	0.0	0.0	8.3	0.0	0.0	0.0
GUI-Owl-1.5-32B	OpenClaw	0.0	0.065	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Qwen 3.7-Plus	Claude Code (baseline)	51.8	0.702	83.3	76.5	29.4	46.7	53.8	66.7	16.7	20.0
Qwen 3.7-Plus	LongHorizon-Harness (ours)	80.7	0.835	88.9	100.0	58.8	73.3	84.6	91.7	66.7	80.0
Change vs. the same backbone on bare Claude Code		+28.9	+0.133	+5.6	+23.5	+29.4	+26.6	+30.8	+25.0	+50.0	+60.0

The first 16 rows are official results reported by WeaveBench (best thinking mode per backbone). All eight domains improve and DOC reaches 100.0. Note that the authors’ runs execute the agent with root privileges inside the task VM, whereas the official baselines run under a regular user account, so the official rows are reference points rather than strictly matched comparisons.

Table 2 · OSWorld 2.0 (108 desktop workflows). Binary is the percentage of tasks whose final benchmark score equals 1; Partial is the mean benchmark score over all 108 tasks.

Agent	Harness / Mode	Binary ↑	Partial ↑
Claude Opus 4.8	Batched actions	20.6	54.8
Claude Opus 4.7	Batched actions	18.2	48.9
GPT-5.5	Batched actions	13.0	49.5
Claude Opus 4.8	Single action	18.5	49.3
Claude Opus 4.7	Single action	13.9	49.1
Claude Sonnet 4.6	Single action	8.3	41.5
MiniMax M3	Single action	4.6	22.3
Kimi 2.6	Single action	4.6	22.1
Qwen 3.7-Plus	Single action (baseline)	2.8	21.5
Qwen 3.7-Plus	LongHorizon-Harness	8.3	35.2

Table 3 · OSWorld 2.0, Opus 4.7 subset (34 tasks).

Agent	Harness / Mode	Binary ↑	Partial ↑
Claude Opus 4.7	Single action	20.6	55.8
Claude Opus 4.7	LongHorizon-Harness	35.3	66.9

Swapping only the harness lifts binary completion by +14.7 points and partial score by +11.1, so the gains are a property of the framework rather than of one backbone. The backbone still decides the quality of each round’s actions, while the harness decides how reliably verified outcomes survive between rounds — that is why the two compound instead of substituting for each other.

Cost & Attribution: Coordination Is Cheap

The manager accounts for only 2.8% (WeaveBench), 2.0% (OSWorld 2.0), and 8.1% (Terminal-Bench 2.1) of total tokens. Auditing is the larger investment at 19.4%, 24.8%, and 38.1% respectively. And overall cost is not uniformly higher: on Terminal-Bench 2.1 the harness consumes 24% fewer tokens than the baseline while gaining 7.5 points.

Two findings sharpen the attribution:

• Overhead is not intrinsic (−33%). On the 17 Games tasks the same framework inverts sign by backbone: Opus 4.7 spends 33% fewer tokens (16.5M → 11.1M) while scoring higher, whereas Qwen 3.7-Plus spends 3.2× more (10.7M → 34.3M). A stronger backbone satisfies a contract in fewer audit–replan rounds.
• Capability is a system property (0.733 vs 0.680). On those same tasks, Qwen 3.7-Plus under the harness reaches 0.733, above the 0.680 that Opus 4.7 reaches on bare Claude Code. A harness cannot add primitive capability, but it decides how much of it survives to the end of the task.

Insights: Four Recurring Patterns in Paired Trajectories

The authors compare paired trajectories in which the Claude Code baseline and LongHorizon-Harness execute the same task using the same Qwen 3.7-Plus model. These cases examine what persists between rounds under the MEA loop.

Case 1 · Recovering from a stalled interaction (0.59 → 0.92)

WEB_task_16 · WebRTC simulcast layer audit. The baseline does notice the failed GUI interaction, but the observation stays buried in its growing execution history, so it retries the same interaction for 400+ steps. The harness writes the stall and the unresolved evidence gaps into the task state, so recovery resumes from the latest audited state rather than from the failed trajectory. Concretely, Wireshark’s “Decode As” dialog stops responding; the baseline degenerates into 400+ steps of repeated clicking, while the auditor rewrites the same difficulty into concrete evidence gaps, and the next round targets only the unmet items.

Case 2 · Auditing apparent completion (0.00 → 0.89)

DOC_task_2 · Heading style normalization. The executor stops at a visually plausible result that misses the specification: the baseline edits the document XML directly, and the task requires the LibreOffice workflow, so it scores 0.00. The auditor re-parses the XML instead, keeping a plausible but non-compliant claim out of the task state. The baseline advances the document’s appearance through direct ODT XML edits and treats “looks right” as done; re-parsing the XML turns completion into an independently checkable claim.

Case 3 · Preserving pre-repair evidence (0.45 → 0.87)

DOC_task_4 · Calc VLOOKUP repair. Part of the required evidence belongs to the state before the repair. The baseline modifies the spreadsheet before that sequence is complete, leaving an inconsistent before-and-after record. Recording the gap as a pending requirement changes the order of work: the original state is documented before the repair may touch it. The baseline does fix the spreadsheet, but its direct-XML repair destroys the original error scene, so its “before” screenshots show post-fix state; the harness locks the pre-repair window first, then audits all nine screenshots for consistency.

Case 4 · Continuing from verified progress (0.53 → 0.85)

WEB_task_10 · Lighthouse performance drill. Both systems can perform the core optimization; they differ in maintaining the rest. The baseline improves the page, but the same session must also drive DevTools, track deliverables and judge its own progress, and the budget runs out. Keeping finished work outside the execution history lets fresh executors spend context on what remains, while the manager maintains continuity across the whole workflow.

Takeaway

Long-horizon capability is a property of the whole model–harness system, not of the model alone. The backbone decides what an agent can do within a single round; the harness decides how much of that survives to the end of the task. That is why the same loop lifts a weaker backbone above a stronger one running bare, and why it cannot help where a task turns on a capability the model simply lacks.

Try It Yourself

The harness is distributed as a Python package (Python 3.10+) and needs one agent runtime on PATH: claude, codex or openclaw.

uv tool install lh-harness   # or: pip install lh-harness

lh-harness run --task "Summarise the files in this directory." \
  --agent claude_code --model qwen3.7-plus --max-rounds 2

Add --dashboard to watch the MEA loop live, or --task @task.md to read the task from a file. Each run is isolated under runs/<run-id>/ with its full audit trail.

Citation

@article{longhorizonharness2026,
  title={LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks},
  author={Ziyu Ma and Hailang Huang and Shun Zou and Yong Wang and Shidong Yang and Yiming Hu and Fei Wei and XiangXiang Chu},
  journal={arXiv preprint arXiv:2608.01964},
  year   = {2026},
  url    = {https://arxiv.org/abs/2608.01964}
}

Source: This article is adapted from the LongHorizon-Harness project page at https://lh-harness.pages.dev/ (AMAP-ML / DreamX team). Paper: arXiv:2608.01964 · Code: github.com/AMAP-ML/LongHorizon-Harness