Weak-to-Strong On-Policy Distillation

Microsoft Research / University of Maryland / MBZUAI

Research official 2 src. ~1 min

Introduces W2S-OPD, which builds a proxy teacher from the logit difference of a weaker positive/negative model pair to distill capability into a student even when no stronger teacher exists. The student minimizes per-position reverse KL against this proxy teacher on its own rollouts.

Why it matters

Tackles a practical limitation of on-policy distillation at the frontier, where no larger teacher model is available to distill from; received 18 upvotes on HuggingFace Daily Papers (2026-08-03).

Importance: 2/5

Notable research contribution to on-policy distillation, but below the 100-upvote HF Daily bump threshold.

distillation llm-training rl weak-to-strong on-policy

Sources

official Weak-to-Strong On-Policy Distillation

official W2S-OPD code repository