TTPO: Test-Time Policy Optimization

Research official 2 src. ~1 min

Label-free post-training method that exploits an asymmetric failure mode in majority-vote pseudo-labels: rollouts that disagree with the pseudo-label are usually wrong regardless of the vote. Distills agreeing rollouts via On-Policy Self-Distillation and penalizes disagreeing rollouts with Grouped RL. Without ground-truth labels, TTPO matches label-supervised OPSD on five competition benchmarks and lifts Qwen3-1.7B from 38.0% to 45.2%.

Why it matters

67 upvotes on HF Daily Papers. Demonstrates a label-free RLVR-style post-training pipeline that competes with label-supervised methods.

Importance: 2/5

official confirmation

post-training reinforcement-learning reasoning test-time-training math

Sources

official HF Daily Papers entry for TTPO (67 upvotes)

official arXiv listing