IdeaGene-Bench: Benchmarking Scientific Lineage Reasoning and Idea Generation
Shanghai Jiao Tong University
IdeaGene-Bench introduces a benchmark representing each scientific paper as typed 'Idea Genome' objects with inheritance, repair, and recombination relations to prior work. The benchmark contains nearly 2,000 lineage traces across 10 scientific domains. The best-performing system reaches only 27.3% exact accuracy on lineage reasoning, revealing a major gap in current LLMs' ability to track how ideas evolve.
Why it matters
Establishes that state-of-the-art models still struggle significantly with compositional reasoning about intellectual provenance — an important signal for knowledge-grounded generation research.
Importance: 2/5
Novel evaluation of scientific reasoning; SOTA models reach only 27.3% accuracy
Sources
official
arXiv paper 2607.08758