HN Debrief

How we measured AI writing across arXiv, and where the measurement breaks

  • AI
  • Science
  • Education
  • Trust & Safety

The post analyzes 12,750 arXiv papers and applies a custom detector meant to estimate how much academic writing now looks machine-generated. The author says the detector was tuned so pre-ChatGPT papers only trip it about 0.4% of the time, then reports a steep rise through 2026, with computer science far above math. The post is careful to frame this as a statistical estimate, not proof that any one paper was written by a model, and it explicitly includes AI-assisted editing in what might get flagged.

Treat these results as a population-level style signal, not evidence about any specific paper or author. If AI-assisted writing is becoming normal in research, the harder problem for teams, reviewers, and publishers is filtering for truth, novelty, and reproducibility rather than policing prose alone.

Discussion mood

Skeptical and uneasy. People broadly believe AI-assisted writing in research is rising, but they do not trust text-only detectors enough to support strong claims about exact rates, and many are more worried about polished garbage overwhelming reviewers than about prose assistance itself.

Key insights

  1. 01

    Aggregate estimates can survive bad single-paper calls

    A detector does not need to be courtroom-grade on individual papers to tell you something useful about a large corpus. If the calibration really keeps pre-ChatGPT false positives near 0.4%, then the right question becomes whether errors stay stable across fields and over time. That is where this result is still vulnerable, because post-LLM human writing may be converging toward model-like style in some disciplines faster than others.

    Use detectors for trend measurement only if you validate them separately by field and by period. A single global threshold is too crude for any organization that wants to compare departments, venues, or product areas.

      Attribution:
    • yorwba #1
    • jasonfarnon #1
  2. 02

    Much of the signal may be AI polishing

    A lot of researchers are not asking models to invent results. They are using them to turn rough, factual drafts into cleaner academic English, especially when writing in a second language or when the genre itself is tedious. That means a detector spike can reflect a real workflow change without implying fraud or even substantive authorship transfer.

    If you set policy around AI writing, separate language polishing from idea generation and citation drafting. Lumping them together will punish legitimate authors and miss the workflows that actually create scientific risk.

      Attribution:
    • lingeringpine #1
    • dopamine_daddy #1
    • vidarh #1
    • bjourne #1
  3. 03

    The bottleneck is reviewer attention, not paper drafting

    The damage from LLM-written papers is not that every AI-assisted paper is false. It is that fluent junk is cheaper to produce than expert review is to consume. Hallucinated citations, AI-rephrased plagiarism, and exaggerated claims all push more low-value submissions into pipelines that already run on scarce reviewer time and institutional trust.

    Plan for defense at the review layer. Stronger citation checks, reproducibility requirements, and higher submission friction will do more than trying to infer authorship from prose.

      Attribution:
    • probably_wrong #1
    • jsrozner #1 #2
    • epq22 #1
  4. 04

    A clean post-2023 control set is missing

    The biggest hole in the measurement is the lack of a modern corpus you can reasonably treat as mostly human-written. Without that, the detector might be learning recent jargon, topic drift, or genre changes instead of AI involvement. Papers about LLMs are an obvious confounder because the vocabulary itself changed dramatically after 2022.

    Before acting on trend lines like this, build control groups that differ in incentives and editorial process, not just publication date. Otherwise you risk mistaking a changing field for a changing author.

      Attribution:
    • jmcqk6 #1
    • arjunvrofficial #1
    • dopamine_daddy #1
    • luciana1u #1
  5. 05

    LLMs break an old quality signal

    Clean, confident academic prose used to signal that someone had at least spent real effort shaping an argument. LLMs preserve that polished surface while weakening its link to understanding, certainty, and accountability. Readers now have to treat well-written text as potentially adversarial persuasion rather than a rough proxy for care.

    Update your internal heuristics for judging written work. Give less credit to polish and more to artifacts that are harder to fake, like data, code, derivations, and precise engagement with prior work.

      Attribution:
    • mrob #1
    • JSR_FDED #1
    • jerf #1
    • fwip #1

Against the grain

  1. 01

    Lower writing friction could unlock real research

    For strong researchers who dislike paper writing, LLMs can remove a genuine bottleneck. If the underlying work is solid, turning diagrams, code, and notes into a first draft may increase the number of worthwhile papers, not just the number of bad ones. The net effect could still be positive even if the gross submission count gets noisier.

    Do not assume every increase in AI-assisted writing is a quality collapse. In your own teams, measure whether writing assistance helps experts publish or document valuable work that previously stayed stuck in notebooks.

      Attribution:
    • alexpotato #1
  2. 02

    Detection may be less useful than better reading norms

    Some readers argued that the exact AI percentage barely matters. Academic publishing already had noise, fraud, and boilerplate before LLMs, and readers should judge claims by evidence rather than by how the prose was produced. From that view, detector scores risk becoming a distraction from the actual job of verification.

    If you are allocating effort, be careful not to overspend on authorship detection at the expense of source validation. Better standards for evidence can matter more than better guesses about who drafted the paragraph.

      Attribution:
    • jrm4 #1 #2
    • zulban #1
  3. 03

    Good prompting can outperform weak human implementation

    One pushback to the anti-LLM tone was that model output often mirrors the clarity of the human directing it. With architecture discussed up front and a review pass from another model, generated code or prose can beat what many humans would have produced on their own. That does not make it trustworthy by default, but it does undercut the idea that all AI-shaped output is obviously worse.

    Judge AI-assisted work by the review process wrapped around it, not by a blanket assumption that it is low quality. Teams using models seriously should standardize planning and cross-checking, not just ban the tools.

      Attribution:
    • OleksandrC #1

In plain english

arXiv ↩
An open online repository where researchers post preprints of scientific papers before or alongside formal peer review.
ChatGPT ↩
A widely used AI chatbot from OpenAI based on large language models.
computer science ↩
The academic field that studies computation, software, algorithms, and related systems, often abbreviated informally as CS.
LaTeX ↩
A document preparation system widely used in academia, especially for papers with math and technical formatting.
LLM ↩
Large language model, a machine learning system trained on large amounts of text to generate and analyze language and code.

Reference links

Detector methodology and replication

Broader framing on AI detection

  • Digital Uncertainty
    Essay arguing that AI detection and generation will co-evolve toward a state where digital artifacts cannot be reliably labeled human or machine.

Detector examples and comparison tools

Reference papers mentioned in testing