Vals AI test finds AI agent teams cost up to 5 times more and rarely build better apps

AI coding tools increasingly let one agent split a job among several helpers working at the same time. Claude Code has subagents and agent teams, and OpenAI's Codex CLI gained an /agents view at DevDay last month. A study published on 9 October by Vals AI, a company that benchmarks AI models, tested whether these teams actually build better apps.

On its benchmark, they usually did not. Teams cost 1.8 to 5.1 times as much as a single agent, and only one of the four team setups scored significantly higher. If you pay for AI usage by the token, in ringgit, that difference adds up.

How the test worked

Vals AI used Vibe Code Bench, its own benchmark in which an AI builds a complete web app from a product specification. The study used 50 apps. Each app was deployed with Docker Compose, then graded by browser agents that clicked through its workflows. The score is the share of each app's interface tests that pass, averaged across all 50.

Two models took part, OpenAI's GPT 6 Sol and Anthropic's Claude Opus 5.5. Each worked alone and as a team, at medium and at maximum reasoning effort, giving eight setups. In team mode, a lead agent could start up to five subagents at a time, and subagents could not start their own.

The results

Ringgit figures are our conversion at about RM4.08 to the US dollar, the rate in early October.

What the numbers show

• One clear team gain. Sol at medium effort scored 7.3 points higher as a team. Vals found the other three differences, ranging from a 0.3-point drop to a 3.4-point gain, were not statistically significant.
• Effort beat teamwork for Sol. Running Sol alone at maximum effort added 11.4 points over Sol alone at medium effort, more than the team did.
• Opus barely moved. Neither a team nor higher effort made a significant difference for Opus. Comparing its most expensive setup, the max-effort team, with its cheapest, a single agent at medium effort, Vals measured a 1.6-point gain at about 30 times the cost, and did not find that gap significant.
• Speed went both ways. Sol's teams finished faster than Sol alone. Opus's teams took longer, up to 2.3 times as long at maximum effort.

How each model ran its team

The two leads worked differently. Sol's lead split each app along its architecture and created all its subagents within about a minute at medium effort. It also ran browser tests itself, which Vals says caught mismatches that only show up when separately built parts meet.

Opus's lead first wrote a shared CONTRACT.md file setting out the database schema, API routes and which agent owned which files. It then started subagents in waves: builders, then feature agents, then testers and reviewers at maximum effort. At maximum effort it used 6.8 subagents per app, against 4.2 at medium.

Sol's team gained most on harder apps, about 15 points against about 1 point on easier ones. Its biggest gains came on apps where the single agent had skipped features.

The limits

Vals points out that each setup ran only once, and the two models ran through different harnesses, so comparisons between models also reflect the harness. It also says the results "may not carry over to other kinds of work". Vals expects teams to help most on jobs that split cleanly into parts that can run side by side, such as large codebases.

What it means if you vibe code

Vals' own conclusion is that "neither option was a clear default", and that model, task and budget should decide. For small web apps like the ones in this test, its numbers point to trying a single agent first, and raising the reasoning effort before paying for a team.

Sources

• Vals AI, Do Agent Teams Pay Off? A Case Study on Vibe Code Bench, 9 October 2026
• Claude Code Docs, Create custom subagents, checked 13 October 2026
• OpenAI, DevDay 2026 Recap, 29 September 2026
• New Straits Times, Ringgit opens higher against major currencies, lower versus greenback, 1 October 2026