SWE-Together: Multi-Turn Benchmark for Coding Agent Evaluation
Togetherbench released SWE-Together (arXiv:2606.29957), a 109-task benchmark that replays real user-agent coding sessions using a reactive user simulator, measuring agents on both correctness and required user correction. Evaluation across Claude Code, Codex, OpenCode, and mini-swe-agent showed Claude Opus 4.8 achieving best pass@1 with fewest corrective interventions. Dataset and evaluation code are open-source at github.com/Togetherbench/SWE-Together.
Why it matters
Standard single-prompt coding benchmarks miss the multi-turn dynamics of real agentic workflows; SWE-Together scores both final correctness and quality of intermediate agent behavior under steering.
Importance: 2/5
Addresses a genuine gap in coding agent evaluation methodology; open-source benchmark covering major coding agents.