ParetoSaddler paper: apply cost preference only at the end of agent harness search

A Nov 1 arXiv paper reports that keeping cost preferences out of intermediate harness-optimization steps produced agent harnesses that scored higher and cost less per task.

If you automatically tune an agent harness for both quality and cost, a paper posted to arXiv on Nov 1, 2026 suggests you may be throwing away good candidates too early. The authors argue that applying your deployment preference, such as favoring cheaper runs, at every intermediate step can reject harnesses that would have led to better ones. Their method, ParetoSaddler, separates candidate promotion from final deployment selection.

Per the arXiv abstract, each iteration has an optimizer update a parent harness into a candidate. Both run repeatedly, and a Bayesian bootstrap over those runs promotes the candidate if it shows credible progress in either quality or cost. Promoted candidates stay available as parents for later steps.

The reported results, from domain-disjoint splits: on ClawMark, quality rose from 70.86% to 81.35% while cost fell from $1.17 to $1.01 per task. On GAIA2, quality went from 56.22% to 62.89% and cost from $0.78 to $0.45 per task. The authors say the Pareto-selected harnesses also beat quality-only optimization on both measures. @omarsar0's summary draws the practical takeaway: if your harness search tracks only accuracy, add repeated runs and a cost axis before deciding what to keep.

What's unclear: we found no released code. The sources we reviewed don't say which harness parts the optimizer edits, how many runs each comparison uses, or what the search itself costs. The idea of keeping a Pareto set is not new to prompt optimization. GEPA keeps a frontier of candidates that each score best on some example, according to Future AGI. That is a different setup from ParetoSaddler's quality-versus-cost trade-off. A DEV Community digest flagged the paper as relevant to anyone automating optimization of their own harness.

Sources