WMRL: scaling automatic research agents via world models
University of Illinois / NVIDIA
The paper identifies environment execution (exclusive sandboxes, real machine time) as the scaling bottleneck in RL post-training of automatic research agents, and replaces it with a learned world model (WMRL). Two corrections — Online Debiasing and Inverse-Variance Denoising — fix the biased, noisy rewards the world model introduces, with proofs that both strictly improve convergence. The authors report a 3-4x training speedup at matched or better final performance; 4B and 9B post-trained agents beat 48B and 120B open-weight agents on held-out benchmarks, and the method transfers to embodied VLA policies.
Why it matters
437 upvotes on HuggingFace Daily Papers (Sep 10) — the top paper of the window; attacks the environment-execution compute wall that will bind agent-RL scaling before GPU supply does.
Importance: 4/5
Notable paper with 437 upvotes on HF Daily (bump for >=100 upvotes)