Position: Stop Automating Peer Review Without Rigorous Evaluation

Large language models offer a tempting solution to address the peer review crisis. This position paper argues that . We ground this positing in an empirical comparison of human- versus AI-generated ICLR 2026 reviews and an evaluation of the effect of automated paper rewriting on different AI reviewers. We identify two critical issues: 1) AI reviewers exhibit a of excessive agreement within and across papers that reduces perspective diversity. 2) AI review scores are trivially gameable through : prompting an LLM to rewrite a paper could significantly increase the scores from AI reviewers, demonstrating that LLM reviewers are easy to game through stylistic changes rather than scientific results. However, non-gameability and review diversity are conditions for automation. We argue that ---not general-purpose LLMs deployed without rigorous evaluation.