TRL v1.11.0 rewrites trl vllm-serve (~10x smaller) and ships AsyncDistillationTrainer
Hugging Face
v1.11.0 (Aug 26) wraps vLLM's own server inside `trl vllm-serve` (~130 lines replacing the previous 1218), measured ~1.5x speedup on GRPO server-mode benchmarks. New experimental AsyncDistillationTrainer with multi-teacher MOPD support; DistillationTrainer gains tool calling; new model/template support for Muse Glimmer, Nemotron 3.5 Lightning, LFM2.5-VL, Qwen3.8, and a DeepSeek-R1-Distill chat template; DAPO/CISPO/VESPO normalizer fix on the Liger path; VLM fixes for GRPO/RLOO. v1.12.0 was an accidental duplicate that has been skipped.
Why it matters
The vllm-serve rewrite dramatically reduces the maintenance surface for GRPO/RLOO trainers and the speedup shows the cost of the previous custom server; AsyncDistillationTrainer adds a multi-teacher path that matters for Mixture-of-Teachers research.
Importance: 2/5
default