Ideas Have Genomes: Frontier LLMs Score Only 27% on Scientific Lineage Reasoning
Shanghai Jiao Tong University
IG-Bench introduces 'Idea Genome objects' — structured representations of research contributions that track how ideas inherit from and diverge from prior work. The dataset covers 1,961 lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains. Frontier LLMs scored only 27.3% on lineage reasoning tasks, revealing a substantial gap between surface-level scientific knowledge and the ability to trace conceptual genealogies. The benchmark also tests lineage-grounded novel idea generation.
Why it matters
Most scientific AI benchmarks test recall of established facts; IG-Bench tests whether models understand how ideas build on each other — a prerequisite for genuine AI-assisted research. The 27.3% ceiling across frontier models, despite strong performance on standard science benchmarks, points to a structural limitation: models lack robust representations of intellectual dependency chains.
Importance: 2/5
New benchmark revealing 27.3% ceiling for frontier LLMs on scientific lineage reasoning