An Empirical Study of Harness Design for Coding Agents
Zoom
The study holds the execution loop fixed and varies three harness components — planning, action space, and context management — across four models and 176 matched settings on SWE-Bench Verified and Terminal-Bench 2.1. Findings: context management matters most under tight budgets (rule-based elision beats LLM summarization), planning mostly cuts cost for strong models, and bash-capable models do well with a bash-only tool interface at much lower cost.
Why it matters
One of the first controlled empirical maps of how scaffolding choices — not just the underlying model — drive coding-agent performance and cost.
Importance: 2/5
Notable empirical study for coding-agent builders