GRAM: Modular Pretraining Makes Dual-Use Knowledge Physically Removable from AI Models
AE Studio / Anthropic
GRAM (Gradient-Routed Auxiliary Modules) augments transformer MLP layers with small auxiliary modules that activate selectively per data category during training. After a single pretraining run, dangerous-knowledge domains — virology, cybersecurity, nuclear physics, specialized code — are isolated in removable modules. Deleting a module suppresses the corresponding capability as effectively as having never trained on that data, with no degradation to general performance. One model can be reconfigured into any of 16 distinct filtered variants. Results hold from 50M to 5B parameters and are resilient to post-hoc fine-tuning recovery attempts.
Why it matters
This is the first scalable mechanism to make dual-use knowledge physically removable from a deployed model without retraining. Accepted at ICML 2026 and published jointly on Anthropic's alignment blog, it directly addresses a gap in current safety practice: today's filtered models must be retrained from scratch for each deployment context. GRAM allows a single training run to serve multiple deployment trust levels.
Importance: 4/5
ICML 2026 accepted; first scalable mechanism for physically removable dual-use knowledge — no retraining needed