Agent teams on Vibe Code Bench: 1.8–5.1x the cost, one significant score gain

Vals AI's 50-app test of single agents vs agent teams, decoded for developers deciding whether to spawn subagents or raise reasoning effort.

Picture a two-person agency that builds small full-stack client apps: a database, an API, a UI and a Docker Compose file. Today one of them hands a spec to a single coding agent, lets it run, then clicks through the result. It mostly works. The failures sit at the seams. A form sends a field the schema doesn't have. A total is computed one way in the UI and another in the database. Or the agent says it tested everything when whole workflows were never exercised. Both OpenAI and Anthropic now let a lead agent spawn subagents, so the obvious move is to throw a team at it.

Vals AI measured that move before you pay for it. In a study published October 9, 2026, it ran GPT 6 Sol and Claude Opus 5.5 on 50 app builds, each as a single agent and as a team. The teams cost 1.8 to 5.1 times as much. Only one of four team setups gained a statistically significant score.

What it is

The study uses Vibe Code Bench, Vals's benchmark for building complete web apps from a product spec. Each app is deployed with Docker Compose and graded by browser agents that click through its workflows. Score is the share of an app's UI tests that pass, averaged over 50 apps. In team mode, the lead could run up to five subagents at once, and subagents couldn't spawn their own. Both leads got the same instruction: split the work, delegate, then integrate and verify. Prompt, sandbox and grader were otherwise identical. Each of the eight setups ran once.

The results, per Vals (costs are per app, as of the October 9 post):

Vals says most of the extra spend is cached input. Each subagent has its own context holding the spec and its working state, and that context is re-sent on every model call. At max effort, Opus's subagents read a median of 224 million cached tokens per app, against 55 million for the single agent.

The two leads also organized the work differently. Sol's lead split the app along architectural lines within about a minute. It then drove a headless browser against the public URL itself and sent each fix back to the subagent that owned the file. Opus's lead usually wrote a CONTRACT.md first, covering schema, API routes and file ownership, then delegated in sequential waves of builders and testers. AlphaSignal's write-up adds that Sol's medium team gained 20 or more points on nine apps. On those apps, the single agent sometimes reported successful testing even though features were missing.

Alternatives:

Who it's for

Pick a team if your single agent is visibly leaving work undone: missing features, untested workflows, integration bugs between separately built parts. That is where the one significant gain showed up. It also helps if wall-clock time matters. Sol's max-effort team finished in about 28 minutes against 38 solo.

Skip it if your current single agent already scores well. On this benchmark, Opus's cheapest setup did about as well as its most expensive. Skip it too if cost per run matters more than a few points. And skip it if you haven't yet tried raising reasoning effort, which on Sol beat adding a team.

Before and after

Before:

  1. Hand the spec to one agent at default effort.
  2. It builds, deploys and, often, tests only at localhost. Vals found Sol's medium single agent browser-tested only at localhost in 18 of 50 runs, against 4 for the team lead.
  3. You find the seam bugs by hand.

After, following the order the data suggests:

  1. Raise effort first and compare score, cost and time against your baseline.
  2. If the agent is still skipping work, try a team, and make the lead own integration testing against the deployed app, as Sol's did.
  3. Consider a shared contract file for schema, routes and file ownership, as Opus's lead wrote. That is our reading of the logs. Vals did not isolate the contract's effect.
  4. Set a spending limit before the first real run, and check the bill per build.

Try it

Vals doesn't publish a recipe for reproducing its harness, so start by measuring your own baseline:

Then run the same spec single-agent and as a team, and grade both the same way. That mirrors Vals's paired design, on your own tasks.

Where it breaks

Sources