Visual Contrastive Self-Distillation
University of Maryland
VCSD removes the need for external teacher models or privileged reasoning traces in self-distillation for vision-language models. An EMA teacher scores tokens under two conditions — with the original image and with the image content erased — and the resulting log-probability gap is used to sharpen the distillation signal, with no added inference-time cost.
Why it matters
Second most-upvoted paper on HuggingFace Daily Papers for this date (25 upvotes), reporting solid gains on Qwen3-VL/Qwen3.5 (e.g. 62.27% to 67.04% on a 2B model) from a teacher-free training recipe.
Importance: 2/5
Notable research release; second-ranked HF Daily Paper, below the 100-upvote bump threshold.
Sources
official
Visual Contrastive Self-Distillation