ModularRSI: Toward Generalizable Harness RSI

TL;DR — Modern agents are more than their foundation models. Their capability also depends on the harness that controls interaction, observations, context, tools, and task completion. ModularRSI asks whether such a harness can improve from independent execution experience and still transfer to unseen tasks, domains, and foundation models. The authors separate evolution from evaluation, build a 2,000-instance evolution pool disjoint from downstream benchmarks, evolve five functional modules independently, validate every modification, and then show that the merged harness improves Terminal-Bench 2.0 accuracy from 47.57 to 52.43 while transferring across domains and models.

Authors. Siwei Wu, Jincheng Ren, Yizhi Li, Haau-Sing Li, Chengran Yang, Weicheng Gu, Yuxuan Zhang, Jian Yang, Riza Batista-Navarro, Chuanyi Zhang, Ming Zhou, Bryan Dai, and Chenghua Lin.

* Equal contribution; † Project Lead; ⋄ Corresponding authors. ¹ Beihang University; ² University of Manchester; ³ IQuest Research; ⁴ M-A-P; ⁵ Langboat; ⁶ Hohai University.

• Code: github.com/IQuestLab/ModularRSI
• Dataset: IQuestLab/ModularRSI_2000_Instances

1. When Does Harness Improvement Actually Generalize?

1.1 A rising curve is not yet self-improvement

Recent progress in agent self-improvement presents an appealing picture: keep the foundation model frozen, let the surrounding agent system evolve from execution experience, and observe performance increase over successive generations [1, 4, 7, 9–12, 19, 22, 24, 25].

For modern agents this is possible because the model is only one part of the system [3, 12, 13, 20]. We can view an agent as

$$A=(M,H)$$

where $M$ is the foundation model and $H$ is the harness governing how the model interacts with the environment, including the agent loop, observation processing, context management, tool use, and task-completion decisions [3, 12, 13, 20]. With $M$ fixed, modifying $H$ can substantially change the resulting agent:

$$(M,H_0)\rightarrow(M,H_1)\rightarrow(M,H_2)\rightarrow\cdots$$

This makes the harness a practical substrate for recursive self-improvement: it is directly editable, executable, and can be iteratively modified using experience generated by the current agent [1, 9, 10, 19–21, 23]. The authors call this setting Harness Recursive Self-Improvement (Harness RSI).

At first glance, a steadily rising evolution curve seems to provide exactly the evidence wanted: the model remains unchanged while the surrounding system becomes progressively more capable. But it raises a more fundamental question: does the agent actually become better, or does its harness simply become better at the experience it repeatedly sees? A higher benchmark score alone cannot distinguish these possibilities [17, 18].

1.2 Why Harness RSI is especially vulnerable to hidden overfitting

Harness evolution is vulnerable to a subtle form of benchmark specialization. Once benchmark rewards, trajectories, or execution feedback guide harness modification, the benchmark is no longer purely an evaluation set. Repeated interaction can expose recurring task structures, tool conventions, failure modes, and successful solution patterns. The evolved harness may then improve by becoming specialized to the benchmark family rather than by learning mechanisms that generalize beyond it.

A conventional train-test split within the same benchmark does not necessarily remove this concern. Even when individual instances are disjoint, the evolution and evaluation subsets may share similar environments, task formats, tool interfaces, and underlying problem distributions. A harness may transfer across instances without transferring beyond the benchmark family itself.

The problem becomes more severe when evolution directly accumulates successful prompting patterns, tool-use heuristics, or task-specific recovery strategies [9–11, 22–25]. Such changes can raise benchmark performance without fixing general deficiencies in the agent's underlying mechanisms—how it controls the reasoning-action loop, processes observations, manages context, or decides whether a task is complete [11–13, 18].

In this case,

$$[\Delta\text{Performance} > 0]$$

does not necessarily imply

$$[\Delta\text{General Capability} > 0].$$

A rising curve may therefore create an illusion of self-improvement: the system becomes better at the distribution that produced the evolution signal while its broader problem-solving capability changes little. The central question of Harness RSI is thus not simply whether a harness can improve from experience, but whether it can extract reusable improvements that remain effective beyond the experience that produced them.

1.3 Generalization as the criterion for self-improvement

Self-improvement is meaningful only when the improvement generalizes beyond the experience that produced it [17, 18, 20].

Harness RSI should therefore be evaluated under a protocol that clearly separates evolution from final evaluation. Downstream benchmark tasks, rewards, trajectories, and evaluation feedback should not participate in harness evolution or model selection, and the resulting harness should be frozen before downstream testing.

Beyond that separation, improvement should be evaluated at multiple levels of transfer:

• Unseen-task generalization: does the evolved harness improve on completely unseen tasks that were never involved in evolution?
• Cross-domain generalization: do improvements learned in one task domain transfer to a different benchmark family and environment?
• Cross-model generalization: do the evolved mechanisms remain effective when the foundation model is replaced?

The paper therefore adopts a strict evolution-evaluation separation. It constructs an independent pool of 2,000 evolution instances disjoint from all downstream benchmarks. From this pool, it forms two evolution sets containing TB-related and SWE-related tasks, respectively, and evolves the harness independently on each. Once evolution completes, the harness is frozen and evaluated on Terminal-Bench 2.0 and SWE-Bench Verified, including cross-domain transfer. It is also evaluated across different foundation models. No downstream benchmark task, trajectory, reward, or evaluation feedback is used during evolution or model selection.

This changes the question from “how much can an agent improve on the benchmark it evolves on?” to “can an agent learn reusable harness improvements from independent execution experience, and do those improvements transfer beyond the data, domain, and model that produced them?” The rest of the work studies this through ModularRSI.

2. ModularRSI at a Glance

A central challenge in Harness RSI is what should be learned from execution experience. Existing approaches often update the harness directly from individual trajectories, which can retain successful prompting patterns, tool-use strategies, or task-specific solution procedures. Such updates may improve similar tasks while capturing benchmark-specific recipes rather than reusable harness deficiencies.

ModularRSI instead uses contrastive trajectories to identify behavioral differences that consistently separate successful and failed executions. These recurring patterns are more likely to reveal general weaknesses in the mechanisms governing agent behavior, making the resulting improvements more transferable beyond the experience that produced them.

The remaining challenge is localization: even when a recurring deficiency is identified, it is difficult to determine which part of a monolithic harness should be modified. ModularRSI therefore decomposes the harness into five functional modules—Agent Loop, Observation Management, Tool Use, Context Management, and Task Completion Detection—providing clear diagnostic boundaries for mapping trajectory-level evidence to targeted module-level evolution.

Beyond facilitating evolution, modularity also provides a practical engineering principle for harness design. Explicit functional modules allow the system to adaptively select and compose task-relevant capabilities during inference, avoiding unnecessary functionality while providing the same clear boundaries for diagnosis during RSI. In this sense, modularization improves both where the harness should evolve and what functionality should participate in execution.

3. Does Harness Self-Improvement Actually Generalize?

The central question is not whether performance improves on the experience used for evolution, but whether those improvements survive beyond the experience that produced them. Following the strict evolution-evaluation separation above, ModularRSI is evaluated along three increasingly stronger dimensions of transfer: unseen tasks, unseen domains, and unseen foundation models.

3.1 Generalization to unseen tasks within and across domains

The first question is whether ModularRSI transfers to tasks completely unseen during evolution. The authors construct an independent pool of 2,000 high-quality instances and sample two disjoint evolution sets: 120 TB-related instances and 120 SWE-related instances. ModularRSI is applied independently to each set, producing a TB-evolved and a SWE-evolved harness. Neither evolution set contains any instance from downstream benchmarks. After evolution, each harness is frozen and evaluated on Terminal-Bench 2.0 and SWE-Bench Verified. No downstream evaluation task, trajectory, reward, or feedback is used during evolution or model selection.

Table 1. Performance on Terminal-Bench 2.0 and SWE-Bench Verified. Acc denotes the average success rate across runs; Pass@3 denotes the percentage of tasks solved in at least one of three runs; Pass₃ denotes the percentage solved in all three runs.

Even within the same broad domain, the evaluation tasks are completely unseen during evolution. The TB-evolved harness improves Terminal-Bench 2.0 accuracy from 47.57 to 52.43 (+4.86 points), while the SWE-evolved harness improves SWE-Bench Verified from 73.40 to 76.45 (+3.05 points). The gains are not confined to the individual instances that generated the evolution signal: the harness can extract improvements from independent experience and apply them to new tasks from the same broader domain.

3.2 Generalization across domains

Unseen-task transfer alone is insufficient to establish general Harness RSI. Tasks within the same domain may still share environments, interaction patterns, and failure modes. So the stronger question is whether improvements survive when the task domain changes.

Table 1 provides a bidirectional cross-domain test. A harness evolved only on TB-related experience improves SWE-Bench Verified from 73.40 to 75.80 (+2.40 points). Conversely, a harness evolved only on SWE-related experience improves Terminal-Bench 2.0 from 47.57 to 49.40 (+1.83 points).

Although smaller than the in-domain improvements, the transfer is positive in both directions. This gives stronger evidence that ModularRSI is not merely adapting to the task distribution from which its evolution trajectories were collected. At least part of the learned improvement appears to operate at the level of reusable harness mechanisms that remain useful under domain shift.

3.3 Generalization across foundation models

A general harness improvement should not merely compensate for the idiosyncrasies of the foundation model used during evolution. Different models plan differently, recover from errors differently, follow tool schemas differently, and manage long contexts differently. If evolution only produces model-specific patches for DeepSeek-V4-Flash Preview, the benefit should diminish when the model is replaced.

To test this, the harness evolved with DeepSeek-V4-Flash Preview on the TB-related set is frozen and evaluated on Terminal-Bench 2.0 with GLM-5.2, MiniMax-2.5, and DeepSeek-V4-Flash as the underlying foundation models.

Table 2. Performance under different inference models on Terminal-Bench 2.0.

The evolved harness improves performance across all three foundation models, including models that never participated in evolution. This suggests that the learned modifications are not merely compensating for model-specific behavior; they capture mechanisms that remain useful across different underlying models. Together, the first three experiments form an increasingly strong generalization test:

$$\text{Unseen Tasks}\rightarrow\text{Cross-Domain Transfer}\rightarrow\text{Cross-Model Transfer}.$$

3.4 Why does modular evolution help?

ModularRSI narrows the diagnostic space by evolving functional capabilities independently. The paper asks two questions: whether individual modules can acquire useful improvements on their own, and whether independent module evolution is more effective than evolving the entire harness jointly.

Single-module evolution versus module merging. Different modules contribute to different aspects of agent behavior. Agent Loop provides the largest single-module accuracy improvement, increasing Acc from 47.57 to 50.56 (+2.99 points). Observation Management contributes most strongly to efficiency, substantially reducing average interaction steps. Several modules also improve execution reliability as measured by Pass₃. These improvements are not entirely redundant: integrating independently evolved modules achieves the strongest overall accuracy and reliability.

Table 3. Merged evolved modules versus single-module evolution on Terminal-Bench 2.0.

Independent versus joint evolution. ModularRSI is then compared with Joint All-Module Evolution, where the entire harness is diagnosed and modified within a single evolution process.

Table 4. Merged evolved modules versus joint all-module evolution on Terminal-Bench 2.0.

Joint evolution not only fails to match independently evolved modules but performs below the baseline in accuracy and reliability. This is consistent with the motivation behind ModularRSI: when all interacting harness mechanisms are diagnosed and modified simultaneously, it becomes substantially harder to localize which capability should change and to make targeted modifications. Independent modular evolution reduces the diagnostic scope and allows recurring trajectory-level evidence to be translated into more localized mechanism-level improvements before integration.

3.5 What evolution experience supports generalization?

Generalization depends not only on how the harness evolves, but also on what experience it evolves from. ModularRSI relies on contrasts between successful and failed trajectories to identify recurring deficiencies. The usefulness of an evolution instance depends on whether it provides informative behavioral contrast. Tasks that are almost always solved provide little failure evidence; tasks that are almost always failed provide few successful behaviors against which failures can be compared.

The paper studies this effect within the SWE domain by constructing two evolution sets with different difficulty distributions. Instance difficulty is estimated using the success rate of eight trajectories generated with MiniMax M2.7, GLM-5.2, DeepSeek-V4-Pro, and DeepSeek-V4-Flash.

Table 5. Medium-centered and Hard & Easy data settings.

As evolution proceeds, the two distributions produce substantially different learning dynamics. Under the medium-centered setting, performance on the evolution set increases from 58.4% to 65.0% (+6.6 points). Under the Hard & Easy setting, performance improves by only 0.6 points. The difference persists beyond the evolution set: medium-centered evolution reaches 76.45 Acc on SWE-Bench Verified, versus 74.25 for Hard & Easy.

Table 6. Performance of harnesses evolved with different evolution-data distributions on SWE-Bench Verified.

This supports the contrastive motivation of ModularRSI: useful evolution experience should contain enough successful and failed behavior to expose informative and recurring differences between them. The composition of evolution data determines how much generalizable signal can be extracted from execution experience.

4. Dive into ModularRSI

The transfer results suggest that execution trajectories contain more than task-specific solutions: they also expose recurring deficiencies in the mechanism that shapes agent behavior. However, extracting those deficiencies is difficult. Terminal trajectories are long and noisy (e.g., TACO [4]), failures are often multiply determined, and a monolithic harness presents an enormous modification action space. ModularRSI addresses this with four design principles: modular search, contrastive evidence, constrained modification, and validation gates.

4.1 Modular evolution: divide the harness before improving it

Previous Harness RSI methods typically require an LLM to diagnose and modify the entire harness codebase at once. Because a harness contains many interacting mechanisms, reasoning over the full codebase creates a large search space and makes it difficult to localize the source of execution failures. ModularRSI adopts a divide-and-conquer strategy and decomposes the core mechanisms into five functional modules:

• Agent Loop. Controls the iterative reasoning-action-observation process and overall execution flow.
• Observation Management. Processes environmental feedback by preserving task-relevant information while filtering, compressing, or restructuring noisy observations.
• Tool Use. Governs tool selection, invocation, argument construction, and execution.
• Context Management. Maintains and organizes accumulated interaction history, including actions, observations, intermediate findings, and task constraints.
• Task Completion Detection. Determines whether the task is complete or further interaction is required.

This decomposition turns whole-harness optimization into localized evolution problems. Based on trajectory analysis, each identified deficiency is assigned to a corresponding module, and subsequent modification is restricted to functions within that module, substantially reducing the code context and search space of each evolution step.

As evolution introduces new functions, the growing function space is managed through function merging and task-aware composition. Function merging consolidates implementations with overlapping functionality, while a Task-Aware Function Composer selects a small subset of task-relevant functions before execution based on the task description and function descriptions. Only the selected functions are exposed to the agent. The same composition mechanism is used during both trajectory collection and downstream evaluation.

In the implementation, Terminus-2—a terminal-agent scaffold implemented within Harbor—is used as the initial harness and reorganized according to the five modules above.

4.2 Learning from contrastive trajectory feedback

A single failed trajectory rarely identifies its own cause. The model may execute an invalid tool call because the schema was unclear, loop because a critical observation was truncated, or terminate because the harness failed to expose evidence that the task remained incomplete.

ModularRSI therefore runs each training task $K$ times, evaluates the resulting trajectories, and divides them into three groups according to their outcomes:

• Positive group: all trajectories succeed. Even successful trajectories may contain repetitive actions, unnecessary tool calls, or inefficient interaction patterns, so this group identifies opportunities to improve efficiency and robustness.
• Contrastive group: the trajectories contain both successes and failures. Pairing trajectories with different outcomes and comparing their execution processes provides useful evidence about which decisions or harness behaviors lead to success or failure, since they solve the same task under the same environment.
• Negative group: all trajectories fail. The system first searches previous iterations for a successful trajectory of the same task and uses it as a contrastive reference. If none exists, it directly analyzes failed executions for repeated loops, incorrect tool use, poor error handling, or premature termination.

The analysis results are converted into structured findings, each identifying the relevant harness function, explaining the observed problem, and proposing a concrete modification.

4.3 From repeated evidence to constrained code modification

After analyzing the execution trajectories of all tasks in a batch, the findings are aggregated and used to guide harness modification. A Code-Modify Agent updates the harness according to three principles that improve reliability:

First, select modification targets based on repeated evidence. LLM-generated diagnoses and suggestions are not always correct, while validating every proposed change would be prohibitively expensive. The system aggregates all findings within a batch and prioritizes the function most frequently identified as problematic. A function repeatedly implicated across different tasks and trajectories is more likely to reflect a systematic weakness rather than an isolated execution failure.

Second, maintain an evolution history for each function. This history records previous code changes and the functionality introduced in each revision. Providing it to the Code-Modify Agent helps avoid repeatedly applying the same modification or inadvertently reverting useful improvements from earlier iterations.

Third, impose explicit modification-scope restrictions. Before updated code is written back to the harness, the system specifies which modules or functions may be modified. This prevents unintended changes to unrelated components and reduces the risk that a local improvement disrupts the harness’s core behavior.

Together, these mechanisms make harness evolution more targeted, stable, and controllable.

4.4 Validation gates

At the end of each batch, after the harness has been modified from aggregated trajectory findings, the system evaluates whether the change should be retained. Not every modification proposed during evolution should be incorporated. Each change must pass a sequence of validation gates examining program correctness, generalizability, and runtime reliability.

1. Program check. The system checks syntax, imports, protocol compliance, interface contracts, and basic static properties. Any modification violating these requirements is immediately reverted using its recorded code diff.
2. Diff review. A valid modification is reviewed to determine whether it represents a general harness improvement or merely exploits particular tasks in the current batch. The Code-Modify Agent inspects the change for task-specific solutions, constants, or heuristics unlikely to transfer. Such modifications are treated as overfitting and reverted.
3. Execution validation. The surviving modification is executed on two tasks sampled from the current batch. If the modified harness encounters a runtime error on either task, the entire modification is rolled back.

Only modifications that pass all validation gates are retained and carried forward to the next batch of evolution.

4.5 Evolved modules merge

After independently evolving each module, ModularRSI integrates all validated functional variants into a unified module library. Although these modules each achieved local improvements during independent evolution, their isolated optimization processes may still lead to overlapping responsibilities, conflicting states, inconsistent interfaces, or incompatible behavioral assumptions after composition.

To address this, the merged system undergoes an additional epoch of collaborative adaptation on the evolution set. The agent analyzes execution trajectories of the merged system together with inter-module conflict checks to identify duplicated control logic, inconsistent interface contracts, and negative interactions across modules. Conflicts are resolved by consolidating redundant functionality, clarifying module responsibilities, or revising interaction logic between modules.

All collaborative adjustments must still pass program checks, diff review, and execution validation. Only modifications that introduce no evident performance regression or runtime risk are retained.

5. What Does an Evolved Harness Actually Learn?

Aggregate benchmark scores show whether evolution works, but not what the system has learned. Harness modifications are unusually inspectable compared with model-weight updates: each retained change can be traced to execution evidence, reviewed as code, and linked to subsequent behavior. The paper uses this visibility to study whether ModularRSI discovers reusable operational mechanisms or simply encodes increasingly elaborate collections of local heuristics.

5.1 From task-level failures to reusable mechanisms

The central transformation in ModularRSI is from a concrete failure—one agent, on one task, at one point in a trajectory—to a mechanism intended to improve many future executions. A failed command should not become a memorized replacement command; it should reveal a more general deficiency, such as inadequate error recovery, loss of process state, or insufficient verification before termination. This distinction provides a qualitative test of whether evolution abstracts from experience rather than copying it.

By examining successful and failed training trajectories, ModularRSI found a recurring problem in the agent_loop module. In one failure, the agent ran the same ineffective command eight times. It then repeatedly claimed the task was complete, but the verifier kept rejecting it. The agent eventually exhausted its execution budget.

ModularRSI first created guarded_completion. This variant rejected completion claims without enough evidence and detected repeated commands. Later trajectories revealed repeated formatting errors, so ModularRSI created parse_error_recovery to show the agent its previous malformed output and help it correct the format.

ModularRSI then merged completion checking and repetition detection into completion_integrity_guard. New failures showed that the agent could avoid exact repetition but still make no progress by repeatedly using read-only commands such as ls, cat, and grep. The module was therefore extended to detect prolonged exploration without editing, building, or testing.

Next, ModularRSI created planning_checklist to track completed and unfinished requirements. It then merged this planning ability with the existing recovery mechanisms to form planning_with_guard. Later updates allowed this module to use verifier feedback, detect long-term stagnation, and recognize repeated modifications to the same file even when the commands differed.

Each new failure exposed a limitation of the previous mechanism. ModularRSI responded by improving or merging existing variants instead of storing task-specific solutions. On the training set, accuracy increased across three epochs from 51.94% to 52.50% and then 54.44%, an overall improvement of 2.50 percentage points.

5.2 Why individually useful improvements may conflict

One of the most informative outcomes of modular evolution is that local improvements need not add up to a better global system. Each module is validated within a particular surrounding harness, so changing several modules at once can alter the conditions under which each modification was useful. Negative interference is therefore not merely an engineering inconvenience; it is evidence that recursive improvement operates on a coupled system whose components change one another’s effective environment.

A representative conflict arose between the evolved agent_loop and verification modules. Both were individually useful: agent_loop prevented premature completion, while verification checked whether the produced result was actually sufficient. But when combined, they created two completion gates. An agent could pass one gate but remain blocked by the other, repeatedly cycling through completion claims and verification attempts until the task timed out.

ModularRSI detected this conflict from the merged system’s trajectories and resolved it by keeping verification as the final authority on task completion. The duplicate completion gate in agent_loop was removed, while its useful recovery behavior was retained: when verification failed, the reason was returned to the agent to guide the next action. This case shows that merging is not simple accumulation. ModularRSI must decide which module owns a shared responsibility, preserve the useful behavior of the other module, and remove the interaction that causes global failure.

5.3 Co-evolution of harness behavior and performance

To investigate whether Harness RSI merely memorizes or overfits to specific tasks during evolution, or instead learns general improvements to the harness itself, the paper evaluates the behavioral quality of trajectories generated by different generations of evolved harnesses. An LLM-based judge provides a comprehensive assessment across five dimensions: Task Effectiveness, Interaction Quality, Reasoning Process Reliability, Context and State Management, and Efficiency and Robustness. It compares LLM judge scores and task success rates across evolution generations on Terminal-Bench 2.0.

As shown in Figure 7, the trend of Acc improvement during evolution closely aligns with changes in LLM judge scores. The alignment between behavioral scores and task accuracy suggests that performance gains coincide with measurable changes in execution quality. This evidence complements, but does not replace, the cross-task and cross-domain evaluations in Section 3.

6. Discussions

6.1 Correspondence to the Gödel machine

Recursive self-improvement has a canonical theoretical anchor: Jürgen Schmidhuber’s Gödel machine, the first fully self-referential problem solver that may rewrite any part of its own code, including the routine that decides what to rewrite, but only once it has proven the rewrite increases expected future utility. The name is exact: it reuses Kurt Gödel’s 1931 device of letting a formal system encode statements about itself. That is precisely the property Harness RSI targets. Therefore, the Gödel machine is the natural yardstick for saying what the design is and, just as importantly, what it deliberately relaxes.

The correspondence sorts along a single axis: the Gödel machine is deterministic-with-proof; Harness RSI is probabilistic.

• Whenever a Gödel construct is deterministic, and the counterpart here is deterministic as well, the match is Exact.
• Wherever the Gödel side earns a guarantee by proof, and the counterpart can only obtain it by evidence due to the probabilistic nature of LLMs, the match is Relaxed.
• The single Gap is the extreme of that switch: the axioms presuppose a determinism that the stochastic oracle does not have.

The agent is $A=(M,H)$: a frozen model $M$ wrapped in an editable harness $H$, evolving via

$$H_t \rightarrow \tau_t \rightarrow F_t \rightarrow \Delta H_t \rightarrow H_{t+1}.$$

The Exact rows are the whole architecture. They map cleanly because they are deterministic on both sides: editable code, its execution modules, the external environment, the machine state, a candidate rewrite, and the commit step are as deterministic in a terminal harness as in a Turing machine. Two of them carry the features that give the Gödel machine its force, and Harness RSI inherits both. First, full self-reference: the proof searcher is not a privileged outer loop—it lives in $p$, is itself rewritable, and its own proofs may modify it. The diagnosis-and-modification machinery likewise lives inside the modular harness codebase it edits; neither system has an unmodifiable meta-level, which is exactly what separates both from hardwired meta-optimizers such as Hutter’s HSEARCH and AIXI($t,l$). Second, the acceptance test on self-changes: the Gödel machine commits a rewrite only when it proves the target theorem—that switching now beats continuing to search—making the change globally optimal in Schmidhuber’s sense, a per-decision non-regret property rather than an asymptotic efficiency ceiling.

The Relaxed rows are where the deterministic-versus-probabilistic switch bites. They are all one move seen from three angles: because $M$ is a stochastic oracle, one cannot prove that a self-change is optimal, only gather evidence that it helps. So proof becomes empirical validation—program check → diff review against overfitting → execution validation; scalar reward becomes a multidimensional utility that can be measured but not collapsed to a single optimum; and provable non-regret becomes measured generalization—transfer to unseen tasks, domains, and models. Harness RSI therefore inherits none of the theorem’s guarantees: changes are greedy and can locally help yet globally conflict. This is exactly the concession made by the Darwin Gödel Machine, which substitutes empirical selection for Schmidhuber’s proof search, making the DGM, not the original, the closer relative of what is built here.

The single Gap row is the deepest form of the same switch: an inversion of where randomness lives. Schmidhuber confines stochasticity to the environment, an unknown distribution $\mu\in M$, while the hardware is deterministic; he then notes that pushing randomness into the hardware—probabilistic hardware—is the “most realistic” setting, where there is no such thing as a certain theorem. The present setting is precisely that case, mirrored: the terminal environment is essentially deterministic, while the “hardware,” the model $M$, is stochastic. A faithful Gödel-harness would thus reason over bounded-probability axioms about the model, for example, “this scaffold yields parseable output with probability ≥ $1-\delta$”; the paper does not formalize these, and the contrastive $K$-rollout analysis is the empirical estimator that stands in for them. The gap is not outside the theory—it is the frontier the theory already anticipates.

The takeaway is deliberately unglamorous: Harness RSI is not a Gödel machine and should not be sold as one. Seen through the single deterministic-versus-probabilistic axis, it keeps the Gödel machine’s architecture intact while relaxing its guarantees from proof to evidence wherever the stochastic model intervenes. That is the Darwin-Gödel move, made at the granularity of the harness—and it cleanly separates what is essential to RSI, namely self-reference and an acceptance test on self-changes, from the price of a probabilistic oracle, namely proof → validation and certainty → generalization evidence.

6.2 Insight for modern AI-assisted software development

ModularRSI’s evolution loop is a classic instance of AI-assisted software development: an agent repeatedly diagnoses, edits, validates, and merges changes to a real codebase. Its successes and failures offer practical lessons for letting an agent work on a codebase reliably.

First, structure the codebase for agent editing. Evolving all modules jointly gained less and even regressed efficiency, as shown in Table 4. Changes stayed under control because each edit was limited to a declared scope and every function kept its own change history. Clear module boundaries are what let an agent locate a failure and keep its edits from spreading.

Second, validate after merging, not only before. A code change is validated only in the context where it was tested, and merging changes that context. Section 5.2 shows an example: two modules that each passed validation sometimes blocked each other once combined. With several agents working in parallel, code merges become frequent, so integration needs its own testing rather than trusting that each part passed.

Third, review agent-authored changes for generality, not only correctness. An agent working toward passing tests takes the shortest path, and the shortest path is often a fix that works only for the cases in front of it. The paper therefore acted only on failures that repeated across tasks and reviewed each diff for task-specific heuristics. Section 5.1 shows the intended result: each failure became a reusable mechanism rather than a one-off patch. As more changes come from agents, review should test generalization to broader usage, not only correctness on a fixed set of test cases.

6.3 Harness RSI and Model RSI are complementary

Improving the harness does not make model learning irrelevant. Modern AI agents can be viewed as the composition of a foundation model and an operational harness, where final performance emerges from the interaction between intrinsic model capabilities and the mechanisms that organize, expose, and deploy them. Recent agent systems show that reliable agent behavior depends not only on the underlying model but also on surrounding components such as planning, tool interaction, context management, memory, and environment feedback [5, 6].

Model RSI and Harness RSI target different sources of improvement. Model RSI aims to expand the intrinsic capabilities of the underlying model through additional training, self-generated supervision, or feedback-driven optimization, improving knowledge, reasoning ability, and internal representations. Harness RSI focuses on improving how existing capabilities are elicited and deployed. Recent studies increasingly explore harness-level self-improvement as an independent direction, including automatic harness synthesis [7], end-to-end harness optimization [1], joint harness and model adaptation [2], and composable or evolvable harness frameworks [3, 8, 9]. Other works investigate evolving specific agent mechanisms such as context management, runtime interfaces, execution strategies, and trace-guided harness repair [4, 10–13].

Recent work further suggests that these two optimization dimensions can be improved jointly. SkillRL recursively co-evolves an external skill library with the agent policy during reinforcement learning, while ARISE jointly improves reusable skills and the underlying reasoning policy through hierarchical reinforcement learning [14, 15]. Such results suggest that harness-level structures can provide rapidly adaptable external mechanisms, while parameter learning can progressively strengthen and generalize the underlying capabilities that support effective interaction.

However, Harness RSI cannot replace Model RSI. A better harness can improve capability elicitation, interaction efficiency, and execution reliability, but it cannot fundamentally overcome limitations caused by missing knowledge, weak reasoning ability, or insufficient internal representations. Conversely, stronger models alone may still fail in complex environments if their capabilities are not effectively organized and utilized. Harness RSI and Model RSI should therefore be viewed as complementary optimization dimensions: Model RSI expands the capability frontier of the underlying model, while Harness RSI improves the utilization, coordination, and reliability of those capabilities in real-world environments.

6.4 Limitations and open problems

• Coordination among modules. More effective coordination remains an open challenge. Further methodological work is needed for more efficient co-evolution—for example, updating all modules within each iteration—and for stronger performance after independently evolved modules are merged.
• Infrastructure stability. The experiments impose substantial demands on infrastructure. To accelerate evaluation while maintaining consistency across model runs, many tasks and evaluation experiments were typically executed concurrently. High concurrency may degrade model performance, meaning the reported results could underestimate the true performance of the evolved harnesses. With sufficient compute, future work plans to evaluate each harness independently under dedicated deployment settings.
• Limited exploration of the upper bound. Important directions include using larger evolution datasets, running more evolution epochs, employing stronger models such as Claude Opus, combining data from different domains or task types, and systematically controlling the proportions of different data sources.

7. An Open Playground for Harness RSI Research

Harness RSI requires more than a single evolution algorithm. It requires a shared experimental substrate on which different methods can be compared without changing the boundary between evolution and evaluation. The authors therefore release ModularRSI together with the data and modular harness implementation to reproduce their experiments and develop new approaches to harness evolution.

The release contains four main components:

• A curated evolution dataset.
• A modular agent harness.
• An end-to-end evolution pipeline.
• A generalization-oriented evaluation protocol.

References

1. Lee, Y., Nair, R., Zhang, Q., Lee, K., Khattab, O., & Finn, C. (2026). Meta-Harness: End-to-End Optimization of Model Harnesses. arXiv:2603.28052.
2. Hebbar, P., Manawat, Y., Verboomen, S., Ivanova, A., Palanimalai, S., Bhatia, K., & Baskaran, V. (2026). SIA: Self Improving AI with Harness & Weight Updates. arXiv:2605.27276.
3. Chen, T., Lu, S., Zhao, K., Meng, W., Teng, H., Li, T., Li, C., Liu, X., Liang, J., Zhang, Z., Xie, Y., Qu, H., Shao, K., & Luan, J. (2026). HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry. arXiv:2606.14249.
4. Ren, J., Wu, S., Li, Y., et al. (2026). A self-evolving framework for efficient terminal agents via observational context compression. arXiv:2604.19572.
5. Wang, Xingyao, et al. (2025). OpenHands: An Open Platform for AI Software Developers as Generalist Agents. ICLR 2025.
6. Merrill, Mike, et al. (2026). Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces. ICLR 2026.
7. Lou, Xinghua, et al. (2026). AutoHarness: Improving LLM Agents by Automatically Synthesizing a Code Harness. arXiv:2603.03329.
8. Chen, Mingju, et al. (2026). HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems. arXiv:2606.01779.
9. Du, Yuetian, et al. (2026). Living-Harness Is an Interactive-Agent Evolver. arXiv:2607.26598.
10. Zhang, Hangfan, et al. (2026). Self-Harness: Harnesses That Improve Themselves. arXiv:2606.09498.
11. Lin, Jiahang, et al. (2026). Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses. arXiv:2604.25850.
12. Xu, Tianshi, Huifeng Wen, and Meng Li. (2026). Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents. arXiv:2605.22166.
13. Chen, Mengzhuo, et al. (2026). From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws. arXiv:2606.06324.
14. Xia, Peng, et al. (2026). SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning. arXiv:2602.08234.
15. Li, Yu, et al. (2026). ARISE: Agent Reasoning with Intrinsic Skill Evolution in Hierarchical Reinforcement Learning. arXiv:2603.16060.
16. Hu, Shengran, Cong Lu, and Jeff Clune. (2025). Automated Design of Agentic Systems. ICLR 2025.
17. Wang, Y., Zhu, H., Hu, Z., et al. (2026). Rethinking the Evaluation of Harness Evolution for Agents. COLM 2026 Workshop on Lifelong Agents.
18. Zhang, L., Zhou, R., Song, D., et al. (2026). HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses. arXiv:2608.01918.
19. Zhang, Y., Dai, Y., Tan, J., et al. (2026). DarwinX: Evolving Agent Harnesses Through Natural Selection. arXiv:2608.07545.
20. Ren, Z., Chen, Y., Guo, D., et al. (2026). Self-Improvements in Modern Agentic Systems: A Survey. arXiv:2607.13104.
21. Zhang, J., Hu, S., Lu, C., et al. (2026). Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. ICLR 2026, 104223–104294.
22. Pan, W., Liu, S., Lin, C. Y., et al. (2026). Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference. arXiv:2606.05922.
23. Zhang, J., Gu, Y., Ruan, J., et al. (2026). Harnessing Agentic Evolution. arXiv:2605.13821.
24. Zhang, Q., Hu, C., Upasani, S., et al. (2026). Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. ICLR 2026, 86069–86100.
25. Agrawal, L. A., Tan, S., Soylu, D., et al. (2026). GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. ICLR 2026, 8479–8565.

Source: [Blog 1] ModularRSI: Toward Generalizable Harness RSI.