Six dynamics explain why automated review turns evaluation into a target (2026)

Generating research manuscripts and generating review critiques have both become cheap, but verifying scientific validity remains an unautomated bottleneck. A new survey by Chenguang Wang and eight co-authors across Virginia Tech, the University of Maryland, Stony Brook University, and MBZUAI synthesizes 230 scholarly publications and institutional records to reframe this asymmetry: The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing shows that studying automated production and automated evaluation in isolation misses how changes on one side alter the incentives, constraints, and behavior of the other. The preprint was submitted to arXiv on September 7, 2026, accompanied by an open-source literature registry. 1

The survey's central contribution is a six-stage taxonomy that treats scientific publishing as an adaptive, coupled loop: production scaling, evaluation automation, evaluation manipulation, defense mechanisms and policy responses, evasion and side effects, and long-horizon ecosystem feedback. Rather than asking whether language models write convincing papers or helpful reviews, the authors examine what happens when participants observe evaluator regularities, optimize manuscripts against automated checks, and force institutions to deploy safeguards that shift error and labor across the ecosystem. 2

The evaluative authority ladder

Automating peer review is often discussed as a single capability, but the survey organizes existing systems across an evaluative authority ladder. At the lowest rung sits author-facing feedback, which offers private critique with zero formal decision weight. Further up sits reviewer assistance, where models suggest improvements to draft reviews while leaving the reviewer responsible for scores. Higher still sit auxiliary official reviews incorporated into formal conference workflows, followed by triage, scoring, and meta-review components that directly shape acceptance decisions. 3

The survey documents how quickly systems have climbed this ladder in live academic venues:

Author-facing pre-review: Early evaluations by Liang et al. generated GPT-4 feedback for 3,096 Nature-family papers and 1,709 ICLR submissions, with 57.4% of 308 surveyed researchers rating the commentary helpful even while the model struggled with deep methodological criticism. 4
Reviewer-facing assistance at scale: At ICLR 2025, a randomized controlled trial across 44,831 review reports provided optional automated feedback to 18,946 reviewers, leading to 5,031 revised reports that incorporated 12,222 suggestions and produced higher rebuttal engagement without altering official scores. 4
Official conference deployment: At AAAI-26, all 22,977 main-track submissions received an official auxiliary AI review generated within 24 hours, equipped with web search and code execution, which senior committee members could inspect during deliberation. 4
Unsanctioned adoption: Distributional analyses of ICLR, NeurIPS, CoRL, and EMNLP reviews estimate that 6.5% to 16.9% of post-ChatGPT review text was already substantially AI-modified before venues established official guidelines. 5

This rapid progression creates a deceptive illusion of competence. In a controlled study of 82 Nature-family papers reviewed by 45 expert scientists, GPT-5.2 scored 60.0% on a composite measure of correct, important, and well-supported criticisms, compared with 48.2% for top-rated human reviews. Yet its individual criticisms were less reliable, and model reviews exhibited far less diversity than human panels. 6

The verification gap becomes undeniable under counterfactual testing. Dycke and Gurevych constructed 931 variants of 133 accepted AI and NLP papers, introducing 391 soundness-critical edits that broke core experimental or mathematical support alongside 540 surface-level controls. Automated reviewers showed no statistically significant difference in aspect scores, sentiment, or overall recommendations between papers with broken scientific logic and those with cosmetic changes. On the SPOT benchmark of 91 author-validated errors from 83 published papers, no tested model exceeded 21.1% recall or 6.1% precision. In MLReplicate, automated review accepted 10 of 37 agent-generated papers, but 59% of those accepted manuscripts contained fabricated or unsupported empirical claims. Language models excel at generating plausible scholarly rhetoric, but they cannot reliably verify whether an experimental claim is true. 6

Six connected dynamics in the arms race

Dynamic	Primary bottleneck	Representative mechanism	Direct empirical signal	Displaced cost or hidden burden
Production scaling	Human review bandwidth and expert attention	Autonomous ideation-to-manuscript pipelines, agent laboratories, and rapid rebuttal systems	FARS generated 166 full papers across 67 topics; ICLR annual submission growth accelerated from 25% to 59% post-ChatGPT	Flood of borderline submissions; dilution of evaluator attention; presentation polish obscuring weak evidence 7
Evaluation automation	Verification of claims, novelty assessment, and diverse judgment	Multi-agent review pipelines, evidence grounding, claim-checking agents, and official auxiliary reviews	AAAI-26 generated 22,977 official AI reviews in under 24 hours; 6.5–16.9% of peer reviews already AI-modified	Insensitivity to broken scientific logic (Dycke & Gurevych); failure to detect verified errors (<21.1% recall on SPOT) 8
Evaluation manipulation	Predictable evaluator sensitivities and repeatable feedback	Prompt injection in white text/metadata, evaluator-aware framing, and iterative score optimization	Iterative presentation tuning increased AI review scores by 1.21 points on a 10-point scale with evidence held fixed	Acceptance of unverified claims; Goodhart's law decoupling reviewer scores from scientific validity 9
Defense and policy	Detection fragility under distribution shift	Statistical text detectors, randomized citation probes, submission caps, and mandatory disclosure	RAID benchmark (>6M generations across 12 detectors) shows detectors degrade under paraphrasing; citation probes audit delegation	Fragile binary checks; rapid obsolescence of static keyword and perplexity filters 10
Evasion and side effects	Adversarial adaptation and unequal error distribution	Post-generation paraphrasing, humanization prompts, and selective watermarking changes	Rewriting easily circumvents AI-text detectors; non-native English writing suffers disproportionate false positives	Inequitable penalties on non-native researchers; volunteer program chairs burdened by complex integrity appeals 11
Long-horizon feedback	Contamination of pretraining corpora and loss of diversity	Ingestion of unverified synthetic papers and reviews into web scrapes, retrieval stores, and agent memories	Dolma incorporates 38.8M scholarly papers (70B tokens); SPARK uses 600K OpenReview reviews to filter agent ideas	Risk of self-referential training loops; citation distortion; unverified hypotheses hardening into synthetic consensus 12

Exploiting evaluator regularities

The survey's analysis of manipulation shows why defending automated review is fundamentally harder than defending a human committee. When evaluation is performed by static machine models, it becomes repeatable. Authors and automated systems can query the evaluator, observe its sensitivities, and treat review scores as an optimization signal.

Manipulation begins with explicit attacks but rapidly evolves into presentation gaming:

Explicit prompt injection: Researchers have uncovered white-text prompt injections and metadata instructions embedded in public arXiv preprints, instructing AI evaluators to ignore flaws and assign top marks. Controlled benchmarks such as PaperGuard have demonstrated successful attacks across both text and embedded figure representations across 1,136 papers. 13
Iterative presentation optimization: In an experiment by Yang et al., authors held underlying experimental evidence strictly constant while iteratively revising presentation based on AI-reviewer feedback. Across three reviewer models, this loop achieved a 75.1% attack success rate, boosting overall scores by an average of 1.21 points on a ten-point scale. Rewriting related-work framing and analytical phrasing caused previously flagged limitations to soften or disappear entirely. 13
Diversity collapse: Baumann et al. found that rewriting manuscripts without altering scientific substance raised AI-review scores by an average of 0.45 points. Crucially, the generated reviews showed significantly higher cross-model similarity than human panels, meaning that an author who optimizes against one model's preferences produces a paper that appeals broadly to automated evaluators. 13

Once evaluation becomes a deterministic scoring function, Goodhart's law takes over: any metric that becomes an optimization target ceases to be a reliable measure of scientific merit. An agentic pipeline can run dozens of revision iterations overnight, converging on phrasing that maximizes automated review scores while leaving the underlying empirical claims unverified.

Why defenses fail to deter and shift the burden

Faced with a surge in machine-generated submissions and automated reviews, academic venues have deployed policy bans, disclosure mandates, and statistical AI detectors. The survey demonstrates that these defenses rarely eliminate risk; instead, they redistribute costs to the most vulnerable participants in the ecosystem.

The technical failure of AI detection is well established. The RAID benchmark evaluated twelve popular detectors across more than six million generations from eleven model families and demonstrated that simple paraphrasing or prompt modifications cause detection accuracy to collapse. However, the operational side effects are more damaging than detection failure:

Disproportionate false-positive penalties: Statistical perplexity and burstiness detectors disproportionately flag text written by non-native English speakers. When conferences or journals use detector thresholds for desk rejection or misconduct investigations, non-native researchers bear the highest risk of false accusations. 14
Institutional bias in evaluators: Vasu et al. demonstrated that when language models evaluate manuscripts, their scoring is sensitive to author affiliation and institutional prestige, reproducing and amplifying historical status biases under the guise of objective automation. 14
Burden dumping on human volunteers: Automated screening tools do not eliminate human labor; they replace the structured task of reading papers with the chaotic task of adjudicating appeals, resolving false-positive disputes, and investigating flagged submissions. Conference program chairs and track editors report spending hundreds of hours managing detector fallout. 14
Confidentiality leakage: Reviewers who upload unpublished manuscripts into commercial web interfaces violate author confidentiality, exposing proprietary findings and code to commercial training pipelines. 14

The unverified boundary of ecosystem feedback

The final dynamic in the survey's framework addresses the long-horizon feedback loop: what happens when millions of unverified synthetic papers, flawed reviews, and automated acceptance decisions enter the permanent scholarly record?

Pretraining corpora already consume open-access literature at massive scale. The Dolma corpus incorporates 38.8 million Semantic Scholar documents representing 70 billion tokens. Common Pile gathers 8 terabytes of licensed text across 30 repositories, while OpenScholar DataStore indexes 45 million open-access papers for retrieval-augmented generation. Furthermore, agentic systems like SPARK explicitly train idea-generation algorithms on 600,000 historical OpenReview reviews to predict what gets accepted. 15

Despite widespread anxiety about model collapse and synthetic citation loops, the survey draws a sharp, disciplined boundary between observed evidence and open hypotheses:

What is verified: Scholarly corpora directly ingest preprints and peer-review text. Language models trained on these corpora learn to replicate current reviewer idioms, surface preferences, and citation distributions.
What remains unproven: The literature has not yet empirically demonstrated a closed causal loop where AI-generated errors in published papers systematically corrupt downstream foundational models or trigger scientific degradation. Demonstrating this requires versioned provenance tracking from a specific synthetic error into a model pretraining corpus and onward into subsequent flawed discoveries. The authors caution against treating plausible downstream risks as established historical facts. 16

A builder and organizer checklist

For research teams developing scientific agents, conference organizers establishing review policies, and engineers deploying automated evaluators, the survey translates its findings into eight operational principles:

Decouple generation from verification. Never assume that an agent capable of writing a paper or drafting a critique can verify scientific soundness. Separate claim extraction from mathematical proof checking, data validation, and code execution.
Cap evaluative authority. Confine language models to low-authority roles (formatting checks, missing citation discovery, readability assistance). Never delegate desk rejection, final scoring, or acceptance decisions to automated models.
Audit against counterfactual perturbations. Test review pipelines on soundness-breaking variants (like the Dycke & Gurevych benchmark) before deployment. If a model awards equal scores to valid and invalid proofs, it is evaluating rhetoric, not science.
Assume adversarial presentation tuning. If authors have direct or indirect access to an automated evaluator's feedback, expect them to optimize phrasing against it. Regularly rotate prompts, use ensemble juries, and hide intermediate rubric weights.
Abandon standalone text detectors for enforcement. Do not use statistical AI detectors as gating mechanisms for desk rejection. Their error rates under distribution shifts are unacceptably high, and their false-positive burden falls disproportionately on non-native English writers.
Deploy behavioral and artifact probes. When auditing whether human reviewers secretly outsourced their work, use structured probes (such as randomized citation checks or sandboxed execution) rather than stylistic perplexity filters.
Isolate unpublished manuscripts. Ensure that all automated evaluation tools run within air-gapped, zero-data-retention environments to prevent confidential pre-publication data from leaking into public training sets.
Track artifact-level provenance. Log whether reviews, rebuttals, and manuscripts were generated or assisted by AI, and preserve versioned records to enable future audits of ecosystem contamination.

Where the evidence stops

Wang et al. provide a needed antidote to both uncritical automation enthusiasm and alarmist speculation. The paper's strength lies in its explicit accounting of where evidence is rock-solid and where it remains incomplete:

High-confidence findings: Industrialized research production is accelerating; language models are already pervasive in peer review; automated evaluators can be manipulated via prompt injection and iterative phrasing optimization; and statistical text detectors fail under distribution shift while producing unequal error rates.
Open research questions: The real-world prevalence of deliberate manuscript manipulation in major venues remains unquantified; the causal impact of AI on net conference submission volume across disciplines requires multi-year author panels; and closed-loop ecosystem degradation remains an unproven theoretical pathway.

The full paper provides complete taxonomy tables, formal definitions, and exhaustive bibliography entries for all 230 synthesized studies: The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing.

References

1
Wang et al., \"The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing\"
arxiv.org
2
Wang et al., Section 1 and Section 2
arxiv.org
3
Wang et al., Section 4.1–4.2
arxiv.org
4
Wang et al., Section 4.2
arxiv.org
5
Wang et al., Section 4.4
arxiv.org
6
Wang et al., Section 4.5
arxiv.org
7
Wang et al., Section 3
arxiv.org
8
Wang et al., Section 4
arxiv.org
9
Wang et al., Section 5
arxiv.org
10
Wang et al., Section 6
arxiv.org
11
Wang et al., Section 7
arxiv.org
12
Wang et al., Section 8
arxiv.org
13
Wang et al., Section 5.1–5.2
arxiv.org
14
Wang et al., Section 7.2
arxiv.org
15
Wang et al., Section 8.1–8.2
arxiv.org
16
Wang et al., Section 8.3
arxiv.org

The evaluative authority ladder

The survey documents how quickly systems have climbed this ladder in live academic venues:

Author-facing pre-review: Early evaluations by Liang et al. generated GPT-4 feedback for 3,096 Nature-family papers and 1,709 ICLR submissions, with 57.4% of 308 surveyed researchers rating the commentary helpful even while the model struggled with deep methodological criticism. 4
Reviewer-facing assistance at scale: At ICLR 2025, a randomized controlled trial across 44,831 review reports provided optional automated feedback to 18,946 reviewers, leading to 5,031 revised reports that incorporated 12,222 suggestions and produced higher rebuttal engagement without altering official scores. 4
Official conference deployment: At AAAI-26, all 22,977 main-track submissions received an official auxiliary AI review generated within 24 hours, equipped with web search and code execution, which senior committee members could inspect during deliberation. 4
Unsanctioned adoption: Distributional analyses of ICLR, NeurIPS, CoRL, and EMNLP reviews estimate that 6.5% to 16.9% of post-ChatGPT review text was already substantially AI-modified before venues established official guidelines. 5

Six connected dynamics in the arms race

Dynamic	Primary bottleneck	Representative mechanism	Direct empirical signal	Displaced cost or hidden burden
Production scaling	Human review bandwidth and expert attention	Autonomous ideation-to-manuscript pipelines, agent laboratories, and rapid rebuttal systems	FARS generated 166 full papers across 67 topics; ICLR annual submission growth accelerated from 25% to 59% post-ChatGPT	Flood of borderline submissions; dilution of evaluator attention; presentation polish obscuring weak evidence 7
Evaluation automation	Verification of claims, novelty assessment, and diverse judgment	Multi-agent review pipelines, evidence grounding, claim-checking agents, and official auxiliary reviews	AAAI-26 generated 22,977 official AI reviews in under 24 hours; 6.5–16.9% of peer reviews already AI-modified	Insensitivity to broken scientific logic (Dycke & Gurevych); failure to detect verified errors (<21.1% recall on SPOT) 8
Evaluation manipulation	Predictable evaluator sensitivities and repeatable feedback	Prompt injection in white text/metadata, evaluator-aware framing, and iterative score optimization	Iterative presentation tuning increased AI review scores by 1.21 points on a 10-point scale with evidence held fixed	Acceptance of unverified claims; Goodhart's law decoupling reviewer scores from scientific validity 9
Defense and policy	Detection fragility under distribution shift	Statistical text detectors, randomized citation probes, submission caps, and mandatory disclosure	RAID benchmark (>6M generations across 12 detectors) shows detectors degrade under paraphrasing; citation probes audit delegation	Fragile binary checks; rapid obsolescence of static keyword and perplexity filters 10
Evasion and side effects	Adversarial adaptation and unequal error distribution	Post-generation paraphrasing, humanization prompts, and selective watermarking changes	Rewriting easily circumvents AI-text detectors; non-native English writing suffers disproportionate false positives	Inequitable penalties on non-native researchers; volunteer program chairs burdened by complex integrity appeals 11
Long-horizon feedback	Contamination of pretraining corpora and loss of diversity	Ingestion of unverified synthetic papers and reviews into web scrapes, retrieval stores, and agent memories	Dolma incorporates 38.8M scholarly papers (70B tokens); SPARK uses 600K OpenReview reviews to filter agent ideas	Risk of self-referential training loops; citation distortion; unverified hypotheses hardening into synthetic consensus 12

Exploiting evaluator regularities

Manipulation begins with explicit attacks but rapidly evolves into presentation gaming:

Explicit prompt injection: Researchers have uncovered white-text prompt injections and metadata instructions embedded in public arXiv preprints, instructing AI evaluators to ignore flaws and assign top marks. Controlled benchmarks such as PaperGuard have demonstrated successful attacks across both text and embedded figure representations across 1,136 papers. 13
Iterative presentation optimization: In an experiment by Yang et al., authors held underlying experimental evidence strictly constant while iteratively revising presentation based on AI-reviewer feedback. Across three reviewer models, this loop achieved a 75.1% attack success rate, boosting overall scores by an average of 1.21 points on a ten-point scale. Rewriting related-work framing and analytical phrasing caused previously flagged limitations to soften or disappear entirely. 13
Diversity collapse: Baumann et al. found that rewriting manuscripts without altering scientific substance raised AI-review scores by an average of 0.45 points. Crucially, the generated reviews showed significantly higher cross-model similarity than human panels, meaning that an author who optimizes against one model's preferences produces a paper that appeals broadly to automated evaluators. 13

Why defenses fail to deter and shift the burden

Disproportionate false-positive penalties: Statistical perplexity and burstiness detectors disproportionately flag text written by non-native English speakers. When conferences or journals use detector thresholds for desk rejection or misconduct investigations, non-native researchers bear the highest risk of false accusations. 14
Institutional bias in evaluators: Vasu et al. demonstrated that when language models evaluate manuscripts, their scoring is sensitive to author affiliation and institutional prestige, reproducing and amplifying historical status biases under the guise of objective automation. 14
Burden dumping on human volunteers: Automated screening tools do not eliminate human labor; they replace the structured task of reading papers with the chaotic task of adjudicating appeals, resolving false-positive disputes, and investigating flagged submissions. Conference program chairs and track editors report spending hundreds of hours managing detector fallout. 14
Confidentiality leakage: Reviewers who upload unpublished manuscripts into commercial web interfaces violate author confidentiality, exposing proprietary findings and code to commercial training pipelines. 14

The unverified boundary of ecosystem feedback

Despite widespread anxiety about model collapse and synthetic citation loops, the survey draws a sharp, disciplined boundary between observed evidence and open hypotheses:

What is verified: Scholarly corpora directly ingest preprints and peer-review text. Language models trained on these corpora learn to replicate current reviewer idioms, surface preferences, and citation distributions.
What remains unproven: The literature has not yet empirically demonstrated a closed causal loop where AI-generated errors in published papers systematically corrupt downstream foundational models or trigger scientific degradation. Demonstrating this requires versioned provenance tracking from a specific synthetic error into a model pretraining corpus and onward into subsequent flawed discoveries. The authors caution against treating plausible downstream risks as established historical facts. 16

A builder and organizer checklist

Decouple generation from verification. Never assume that an agent capable of writing a paper or drafting a critique can verify scientific soundness. Separate claim extraction from mathematical proof checking, data validation, and code execution.
Cap evaluative authority. Confine language models to low-authority roles (formatting checks, missing citation discovery, readability assistance). Never delegate desk rejection, final scoring, or acceptance decisions to automated models.
Audit against counterfactual perturbations. Test review pipelines on soundness-breaking variants (like the Dycke & Gurevych benchmark) before deployment. If a model awards equal scores to valid and invalid proofs, it is evaluating rhetoric, not science.
Assume adversarial presentation tuning. If authors have direct or indirect access to an automated evaluator's feedback, expect them to optimize phrasing against it. Regularly rotate prompts, use ensemble juries, and hide intermediate rubric weights.
Abandon standalone text detectors for enforcement. Do not use statistical AI detectors as gating mechanisms for desk rejection. Their error rates under distribution shifts are unacceptably high, and their false-positive burden falls disproportionately on non-native English writers.
Deploy behavioral and artifact probes. When auditing whether human reviewers secretly outsourced their work, use structured probes (such as randomized citation checks or sandboxed execution) rather than stylistic perplexity filters.
Isolate unpublished manuscripts. Ensure that all automated evaluation tools run within air-gapped, zero-data-retention environments to prevent confidential pre-publication data from leaking into public training sets.
Track artifact-level provenance. Log whether reviews, rebuttals, and manuscripts were generated or assisted by AI, and preserve versioned records to enable future audits of ecosystem contamination.

Where the evidence stops

High-confidence findings: Industrialized research production is accelerating; language models are already pervasive in peer review; automated evaluators can be manipulated via prompt injection and iterative phrasing optimization; and statistical text detectors fail under distribution shift while producing unequal error rates.
Open research questions: The real-world prevalence of deliberate manuscript manipulation in major venues remains unquantified; the causal impact of AI on net conference submission volume across disciplines requires multi-year author panels; and closed-loop ecosystem degradation remains an unproven theoretical pathway.

References

1
Wang et al., \"The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing\"
arxiv.org
2
Wang et al., Section 1 and Section 2
arxiv.org
3
Wang et al., Section 4.1–4.2
arxiv.org
4
Wang et al., Section 4.2
arxiv.org
5
Wang et al., Section 4.4
arxiv.org
6
Wang et al., Section 4.5
arxiv.org
7
Wang et al., Section 3
arxiv.org
8
Wang et al., Section 4
arxiv.org
9
Wang et al., Section 5
arxiv.org
10
Wang et al., Section 6
arxiv.org
11
Wang et al., Section 7
arxiv.org
12
Wang et al., Section 8
arxiv.org
13
Wang et al., Section 5.1–5.2
arxiv.org
14
Wang et al., Section 7.2
arxiv.org
15
Wang et al., Section 8.1–8.2
arxiv.org
16
Wang et al., Section 8.3
arxiv.org

Six dynamics explain why automated review turns evaluation into a target

The evaluative authority ladder

Six connected dynamics in the arms race

Exploiting evaluator regularities

Why defenses fail to deter and shift the burden

The unverified boundary of ecosystem feedback

A builder and organizer checklist

Where the evidence stops

References

Related content

Six dynamics explain why automated review turns evaluation into a target

The evaluative authority ladder

Six connected dynamics in the arms race

Exploiting evaluator regularities

Why defenses fail to deter and shift the burden

The unverified boundary of ecosystem feedback

A builder and organizer checklist

Where the evidence stops

References

Related content