AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
Microsoft
Frames harness improvement as an offline learning problem and iteratively updates the harness from failure signals in mini-batches. Combines failure-trace diagnosis, structured patch generation treating the harness as code, and validation-based update selection. Across GAIA2 / SWE-Bench Pro / Terminal-Bench 2.0, gains are 9.0 / 9.6 / 10.0 percentage points respectively. Ablations isolate three effective ingredients: deep debugging rather than shallow reflection, targeted rather than unconstrained edits, and generalization-aware rather than trajectory-specific selection.
Why it matters
HF Daily 33 upvotes on Aug 26. From Microsoft: automatic harness engineering is a leverage point for agent reliability, and the ablation isolates the specific design moves (deep debugging, targeted patches, generalization-aware selection) that actually move the needle.
Importance: 3/5
HF Daily 33 upvotes