AgentOPSD: recursive self-distillation for agentic reinforcement learning

Researchers from Tsinghua, Zhejiang University and Meituan propose AgentOPSD, a method that redistributes sparse outcome rewards into turn-level credit signals for long-horizon multi-turn agent RL, using recursive Bayesian updates over teacher-student likelihood gaps instead of uniform trajectory-level advantage broadcasting (as in GRPO). It reaches 89.1% success on ALFWorld with Qwen2.5-7B without extra rollouts or learned critics.