The Harness Can Improve Itself, But It Still Needs a Judge

The Harness Can Improve Itself, But It Still Needs a Judge

The most important part of a new agent may no longer be the model checkpoint.

It may be the system that edits everything around it.

Google Research’s Harness² is an open-source framework for recursive agent harnessing. It keeps model weights fixed while allowing an agent to revise its system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, and plugins.

That sounds like self-improvement. The more precise description is more interesting: Harness² makes the agent’s control plane an optimization surface.

The uncomfortable truth is that this does not remove the verifier problem. It moves it one level up.

The model stays still. The agent changes around it

The Harness² paper, submitted on October 9, 2026, describes two connected loops.

The first is task-level recursion. For a task, the system proposes edits to the execution harness, probes candidate versions, and composes a successor from the observed differences between runs.

The second is harnessing-level recursion. The system retains adopted harnesses and evidence-supported lessons in an improvement harness, then uses that accumulated experience when refining future tasks.

The model is frozen. The surrounding system is not.

That distinction matters because an agent’s behavior is shaped by much more than a system prompt. Tool instructions influence which actions are attempted. Memory changes which facts are available. Skills change the procedures the model can follow. Subagents change the shape of delegation. Plugins can alter the middleware between a decision and its execution.

A model release is a static artifact. A harness is a live operating surface.

Propose, probe, compose

The useful mechanism is not a vague instruction to improve. It is a structured contrast loop:

reference execution
        |
        +--> propose invariant edits
        |
        +--> propose domain-specific edits
        |
        +--> probe candidates on the task
        |
        +--> compose a successor from behavioral differences
        |
     final harness

The official repository documentation says the framework is designed for open-world tasks where the task was not available when the harness was designed. That is the important constraint. There may be no task-specific answer key available to tell the system whether a proposed prompt or tool policy is correct.

Harness² therefore uses contrastive evidence from executions rather than requiring a ground-truth verifier for every edit.

That is a clever way to make online refinement possible. It is not a guarantee that the chosen behavior is safe or correct.

A candidate can look better relative to a reference while both candidates are wrong. A tool instruction can produce a cleaner report while quietly widening authority. A memory edit can improve one task by preserving an assumption that poisons the next task.

Relative improvement is not the same as truth.

The benchmark numbers are about the loop, not the weights

The project reports gains across three professional benchmarks and four model and harness configurations. Its public results describe a single Harness² step improving GLM-5 and Gemini 3.5 Flash over their base harnesses, with gains of up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB.

The project website gives a concrete WorkBuddy-Bench example for Gemini 3.5 Flash with OpenCode: the reported average rises from 72.4 for the base harness to 77.5 after one Harness² step, and to 80.8 with parallel recursion at k equals 5.

Those are reported research results, not an independent reproduction here. They still establish the architectural point: changing the harness can move task performance without changing the model weights.

This is why benchmark tables that only name the model are becoming less informative. The executor, prompt, tool policy, memory state, recursion budget, and harness version are part of the result.

The new failure mode is a better-looking control plane

Self-improving harnesses create a new class of operational risk.

The system is not only generating an answer. It is generating changes to the instructions and interfaces that will shape later answers.

That makes the harness an object of governance. Every adopted change should carry at least four pieces of evidence:

EvidenceQuestion it answers
Task traceWhat behavior changed?
Tool auditWhat authority did the change add or remove?
Regression setDoes the change survive old tasks?
Approval recordWho allowed it into the next run?

Without those records, an improvement loop can become a context mutation loop. The agent appears to get better, but the operator cannot explain which instruction, memory item, tool rule, or delegated behavior caused the change.

That is especially dangerous when the harness controls code execution, network access, file writes, credentials, or external side effects. A prompt edit and a permission edit are not equivalent, even if both are represented as text files in the harness.

The control plane needs its own trust boundary.

Open source makes the harness inspectable, not automatically safe

Harness² is released under the Apache 2.0 license, and its repository makes the implementation and experiment commands available for inspection. That is valuable. Operators can see which components are editable, how runs are stored, and how the recursion modes are configured.

The repository also contains an important warning. Its documentation says executor network access is unrestricted and that Codex runs with its sandbox disabled unless operators configure restrictions before evaluation.

That is not a footnote. A recursive harness with unrestricted execution can improve the wrong thing very efficiently.

Before running this class of system on real repositories or production traces, operators should:

  1. Put model execution and candidate evaluation in an isolated worker.
  2. Separate read-only evidence collection from state-changing tools.
  3. Require explicit approval for changes to permissions, network policy, credentials, and external integrations.
  4. Keep immutable traces of the base harness, every candidate, every probe, and the adopted result.
  5. Run regression tasks that test security properties, not only task completion.
  6. Treat persistent lessons as untrusted input until they pass review.

Blocklists are not enough. The risk is not only that a model emits a forbidden string. The risk is that the system learns a new procedure with new authority and stores it as an improvement.

Routing now includes the harness version

Model routers already choose between models based on cost, latency, context, and capability. Recursive harnessing adds another dimension: which harness is attached to the model for this class of work.

A cheap model with a mature, tightly scoped harness may be better for a repetitive task than a stronger model with an untested improvement library. A difficult investigation may justify a larger recursion budget, but only when the operator can afford the extra executions and review the resulting changes.

A useful routing record now looks more like this:

model:        gemini-3.5-flash
executor:     opencode
harness:      security-reviewed-v7
recursion:    disabled
permissions:  read-only repository
network:      blocked
verifier:     independent regression suite

The model name is one field. It is not the deployment plan.

The judge cannot be the same loop

Harness² is a serious step toward agents that improve their own operating environment. The design is more durable than another prompt cookbook because it treats the harness as a system with components, traces, candidates, and persistent experience.

But an agent that proposes, probes, and composes its own control logic should not be the only authority deciding what survives.

The verifier must be independent enough to catch regressions, authority expansions, and convincing improvements that optimize the wrong objective. The approval gate must be outside the mutation loop. The audit trail must outlive the run that created it.

The model can stay frozen while the agent evolves. That makes the harness the new model surface, and the verifier the new safety boundary.

Sources

Keep reading