Continual Learning Mechanisms Compose for Long-Horizon Memorization
Johns Hopkins University
The paper formalizes long-horizon memorization: a model learns 100 sequential query-answer tasks via continual fine-tuning without retaining examples. No single continual-learning mechanism prevents catastrophic forgetting, but composing complementary mechanisms (data/function/weight anchors with merged LoRA) lifts average retention from 1.2% to 34.9% — a roughly 28-fold gain.
Why it matters
Top paper on HuggingFace Daily Papers for 2026-09-16 with 287 upvotes; a systematic result that mechanism composition, not any single technique, is the path to models that keep knowledge learned over time.
Importance: 4/5
Notable paper + 287 upvotes on HF Daily Papers (+1 bump)