InnoEval and new benchmarks show AI models struggle with original research

Frontier AI models can summarize a thousand papers before your coffee cools. Ask them to invent a genuinely new research technique from scratch, and the results get a lot less impressive. That is the shared conclusion of a cluster of studies from 2025 and 2026, led by a new evaluation framework called InnoEval. Together, they suggest AI models underperform when asked to develop new research methods without prior information to lean on. What the research actually found InnoEval first appeared in an arXiv paper in February 2026 and was later presented at ICML 2026. Its goal is to measure innovation in a structured way, using what its authors describe as knowledge-grounded, multi-perspective evaluations. That approach paid off on its own terms. InnoEval beat traditional LLM-as-judge methods by up to 16.18% in F1 scores on specific tasks. Better measurement, however, produced some unflattering readings. The more carefully researchers looked, the less originality they found. Take the “Reconstruction” benchmark. It tested whether AI models could recover the core idea of a paper using only the reference list that existed before publication. Across 643 papers, as of August 2026, models rec...