PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking
Shows that extra test-time iterations in looped language models do not by themselves help on longer problems, and fixes this by training each iteration to predict the state after the matching number of operations while rescaling recurrent states to their first-iteration norms. On swap-S5, a Qwen2.5-0.5B model trained on at most 32 moves reaches 95.5% mean full-state accuracy at 512 moves, against 55.7% for RLT and 0.8% for explicit chain of thought.
Why it matters
At 158 upvotes it delivers a 16x length-generalization jump from a 0.5B model, evidence that looped architectures can be trained into algorithmic extrapolation rather than merely scaled into it.
Importance: 4/5
158 Hugging Face Daily upvotes and a 16x length-generalization result