LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
Mind Lab
LongStraw is an execution framework for reinforcement-learning post-training on prompts and rollouts spanning millions of tokens under fixed GPU memory. It uses Group Relative Policy Optimization, skips gradient tracking on shared prompt prefixes, and replays response branches sequentially, demonstrating processing of 2.1M token positions on H20 GPUs.
Why it matters
Addresses a widening gap between how far models can already reason/retrieve at inference time versus how far RL training pipelines can actually train on, which has been a practical bottleneck for scaling long-horizon agentic RL.
Importance: 2/5
Notable research paper