Program-as-Weights: Compiling Task Specs into LoRA Adapters for On-Device Inference
University of Waterloo / Harvard
A 4B-parameter compiler trained on a 10M-example dataset (FuzzyBench) translates natural-language task specifications into compact LoRA adapters. The adapter runs on a frozen 0.6B Qwen3 interpreter and matches Qwen3-32B direct prompting on tasks like log-line triage and intent-ranked search, using roughly 1/50th the inference memory at 30 tokens/s on a MacBook M3.
Why it matters
Challenges the assumption that stronger AI always requires larger models at runtime; compile-once-run-offline enables capable narrow-task execution on consumer hardware without cloud inference. Paper reached 86 upvotes on HF Daily Papers, the highest of the week.
Importance: 3/5
HF Daily Papers top paper (86 upvotes); 50x memory reduction with matched quality is immediately reproducible.