Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE
MIT / NVIDIA
Jet-Long is a tuning-free method for extending LLM context windows beyond pretraining limits using a dual-window RoPE: a local window faithful to the original RoPE and a long-range window with a rescaling factor that adapts dynamically to the current sequence length. Achieves up to 1.39× throughput improvement over baseline long-context methods, strong results on RULER and HELMET-RAG benchmarks, and generalizes to hybrid attention architectures without retraining. Extrapolates to 128K tokens.
Why it matters
Long-context extension typically trades off throughput or short-context fidelity. Jet-Long's dynamic bifocal approach avoids both penalties without fine-tuning, making it immediately deployable on top of existing models for RAG and document-level tasks.
Importance: 2/5
Tuning-free long-context RoPE extension to 128K from MIT/NVIDIA with 1.39× throughput improvement and no short-context regression