SelfSearch: a coding agent rewrites its own harness, claims Codex parity for $4.03

A research method where coding agents edit their own prompts, tools and procedures without task rewards, for builders who maintain their own agent harness.

Picture a platform engineer at a 30-person SaaS company. She runs a coding agent on a cheap open-weight model to clear CI chores: bumping dependencies, fixing flaky tests, writing small migrations. The model is fine. The harness around it is the problem: the system prompt, the tool definitions, the "run tests before you commit" procedure. It's a pile of edits she made one Friday at a time. Each tweak means rerunning a batch of tasks to see if anything moved, which costs tokens and an afternoon. Worse, every improvement is tuned to whichever tasks she happened to test. Three weeks later, nobody remembers why the shell tool has that odd timeout flag.

That's the gap a new paper is aimed at: SelfSearch: Reward-Free Search for Self-Improving Agents, posted to arXiv on 2026-09-29. The idea is to let the agent edit its own instructions, tools and execution procedures. It learns from a written record of its earlier edit attempts, not from repeated benchmark runs.

The headline claim, highlighted by @omarsar0 on 2026-10-05, is that $4.03 of search produced a harness that solves 82.0% of Terminal-Bench 2.1 with DeepSeek V4 Flash. That matches Codex, the top harness in a public nine-harness comparison. The claim is interesting. It also needs context.

What it is

SelfSearch is a research method described in an arXiv paper. It is not a shipped tool. Per the authors' abstract, it has three moving parts, and each maps to a pain in the scenario above.

The results below are the authors' claims, from the abstract:

The alternatives, each with its trade-off:

Who it's for

Pick this up if you maintain a custom harness of your own: your own prompt, tool layer and agent loop around a cheap model. It also fits if you already treat that harness as code you iterate on, and you're comfortable reimplementing an idea from a paper. This is the vibe-coding-to-vibe-engineering move applied to the agent itself. The scaffolding is something you improve on purpose, with a record, not by feel.

Skip it in three cases:

Before and after

Before (hand-tuned harness):

  1. Notice the agent failing a class of task.
  2. Edit the system prompt or a tool definition by hand.
  3. Rerun an eval batch and pay for it.
  4. Keep or revert the change. The reasoning lives in your head or a terse commit message.
  5. Repeat. Over time the harness bends toward the tasks you happened to test.

After (SelfSearch, as the abstract describes it):

  1. Start from an initial agent.
  2. The agent modifies its own instructions, tools and procedures.
  3. Each modification attempt is recorded with its reasoning, tool actions and outcomes.
  4. The modified agent, with those records, becomes the next improver. No task reward is consulted during this loop.
  5. The result is a population of evolved agents. The abstract reports their downstream success after the fact.

Our reading: step 5 still implies some evaluation to choose which evolved harness you actually ship. The abstract doesn't say how that selection works, or whether its cost is part of the $4.03.

Try it

The sources don't describe a way to run SelfSearch today. Research found nothing beyond the arXiv abstract: no released code and no published evolved harness. The smallest real step is to read the paper (PDF and HTML versions are on the arXiv page).

You can borrow the core idea without the paper's code. Start keeping a structured record of each harness change: what you changed, why, and what happened. That is our suggestion, not a procedure from the paper. Still, those records are the raw material SelfSearch's agents learn from.

If you want to experiment on DeepSeek models, three facts matter:

Where it breaks

"Matches Codex" is a modest bar. DeepSeek's own changelog lists V4 Flash at 82.7 on Terminal-Bench 2.1 using its DeepSeek Harness minimal mode. Whether those settings match the paper's is unknown, so treat the comparison as rough. Still, it suggests 82.0% on that model is close to what a vendor harness already reported.

The model has also moved on. The same changelog lists V4.1 Flash at 90.6. Latent Space's AINews roundup counts DeepSeek V4.1 Flash among the open models that are generally ahead.

Harness noise is the same size as the claim. Distk's read of DeepSeek's scaffold table finds a five-point swing on the same model depending on the harness. Distk treats one- to two-point gaps as "ties, not wins." A harness that "matches" the top score sits inside that band.

The comparison isn't pinned down. We couldn't confirm which public nine-harness comparison the paper used. DeepSeek Agent's guide says DeepSeek evaluated V4.1 Flash across eight scaffolds, including Claude Code, Codex, OpenCode, Pi, mini-SWE and three DeepSeek Harness modes. Whether that is related to the paper's comparison is unknown. Also unconfirmed:

The cost figures are narrower than they look. It's unclear whether $4.03 includes final evaluation or only the search. The 38.5% execution-cost cut applies only to tasks both agents solved.

Benchmark choice. Distk describes Terminal-Bench 2.1 as the "established set." It notes that V4.1 Flash trails by wide margins on the harder 3.0 and 4.0 versions. The SelfSearch abstract reports no results on those newer versions.

Unaddressed failure modes. No source we found covers:

Our analysis: an agent that rewrites its own tool layer produces code you'd want to review like any other pull request. The abstract says nothing about guardrails.

Portability is unshown. Nothing in the sources demonstrates SelfSearch on Claude Code or Codex setups. We also found no sources comparing it with DSPy-style prompt optimization or skills-based approaches. For now, it's a promising argument for treating your harness as iterable code, backed by one abstract and vendor-reported baselines.

Sources