Benchmarks

The same tasks run through every harness — what a finished task actually costs, and how often it finishes.

Long Horizon Terminal Bench

Multi-hour terminal tasks run unattended in real sandboxes — five harnesses across three completion windows, one fixed model throughout.

August 2026

Most tasks solved

13 / 16 (81.3%)

Vetta · priority

Cost / successful task

$0.1709

Vetta · loose

Field

5 harnesses · 3 windows

Full results →

SWE-Bench Pro

Real repository issues resolved end-to-end — five harnesses on the same fifteen-task slice with GLM-5.3-Flash throughout.

September 2026

Most tasks solved

15 / 15 (100.0%)

Claude Code · GLM-5.3-Flash

Cost / successful task

$0.0398

Vetta · GLM-5.3-Flash

Field

5 harnesses · 15 tasks

Full results →

Business Bench

Spreadsheets, bookkeeping, reports, extraction, drafting, and tooling — 25 matched business operations tasks and one fixed model.

September 2026

Cost / successful task

$0.0223

13% less than Stock Pi

Tasks solved

12 / 25 (48.0%)

Vetta

Field

5 harnesses · 25 tasks

Full results →

TheAgentCompany

Workplace tasks spanning code, chat, documents, spreadsheets, and project tools — ten matched tasks and one fixed model.

September 2026

Cost / successful task

$0.0225

26% less than Codex

Tasks solved

6 / 10 (60.0%)

Vetta

Field

5 harnesses · 10 tasks

Full results →

PRDBench

Product requirements implemented and judged criterion by criterion — five matched tasks, one model, and a 45-minute limit.

September 2026

Cost / successful task

$0.0194

7% less than Codex

Tasks solved

4 / 5 (80.0%)

Vetta

Field

4 of 5 harnesses · 5 tasks

Full results →

YC-Bench

A startup run autonomously for a simulated year — hiring, research, payroll, deadlines, and bankruptcy risk across three seeds.

September 2026

Cost / successful task

$0.825

29% less than Stock Pi

Tasks solved

3 / 3 (100%)

Vetta

Field

5 harnesses · 3 seeds

Full results →

CoffeeBench

A roaster operated inside a six-company agent economy — sourcing, negotiation, inventory, sales, and cash discipline.

September 2026

Cost / successful task

$0.337

Tuned build, different seeds

Tasks solved

5 / 5 (100%)

Vetta

Field

4 harnesses · 5 runs each

Full results →

Further studies publish here once every cell clears the same verification bar.