On-Policy Self-Distillation in Diffusion Models

DiffusionOPSD converts image-level reward guidance into explicit targets for clean-output predictions at sampled queries, separating target construction from finite realization. A frozen behavior policy generates trajectories and supplies query states and anchors; reward gradients construct bounded positive and negative targets around each anchor; the trainable policy fits these as detached supervision via finite fitting before an EMA refresh of the behavior policy. Achieves best final held-out scores in 19 of 20 reward-matched settings across SD 3.5-M and Z-Image-Turbo and ten evaluators, outperforming the strongest competitor by up to 44%, with 40%/63% training-GPU-hour reductions versus DiffusionNFT.