Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation
Improves policy-shift distillation by using an intermediate checkpoint of the same RL run as a waypoint: early and late changes in the teacher's relative action preferences are compared and the endpoint shift is gated (suppressed) when they point in opposite directions. Costs only one extra frozen-teacher scoring pass and wins in all six domain-student settings across Qwen3-8B/14B/30B-A3B students on math and code reasoning.
Importance: 3/5
notable release (major version, significant open-weights drop, or regulatory shift)
Sources
official
arXiv:2610.21601 — Waypoint-OPD