Skip to content
The methodOpen data
Back to the collection

Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews

Weixin Liang, Zachary Izzo, Yaohui Zhang

Show all 12 authors

Haley Lepp, Hancheng Cao, Xuandong Zhao, Lingjiao Chen, Haotian Ye, Sheng Liu, Zhi Huang, Daniel A. McFarland, James Y. Zou

2024 · arXiv

Science & researchLanguage modelsPsychology
Read the paper
HN score
0
Mentions
4
Readers
4
Months
3
What’s in a score?

From the abstract

We present an approach for estimating the fraction of text in a large corpus which is likely to be substantially modified or produced by a large language model (LLM). Our maximum likelihood model leverages expert-written and AI-generated reference texts to accurately and efficiently examine real-world LLM-use at the corpus level. We apply this approach to a case study of scientific peer review in AI conferences that took place after the release of ChatGPT: ICLR 2024, NeurIPS 2023, CoRL 2023 and EMNLP 2023. Our results suggest that between 6.5% and 16.9% of text submitted as peer reviews to these conferences could have been substantially modified by LLMs, i.e. beyond spell-checking or minor writing updates. The circumstances in which generated text occurs offer insight into user behavior: the estimated fraction of LLM-generated text is higher in reviews which report lower confidence, were submitted close to the deadline, and from reviewers who are less likely to respond to author rebuttals. We also observe corpus-level trends in generated text which may be too subtle to detect at the individual level, and discuss the implications of such trends on peer review. We call for future interdisciplinary work to examine how LLM use is changing our information and knowledge practices.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

4 selected excerpts · 4 total mentions

jean-
Neutral mention · rule
Great article. One of the papers it cites is https://arxiv.org/abs/2403.07183, which is also great and looks at the issue of LLM usage to write peer reviews. It’s an issue I’ve noticed personally, as I’m seeing an increasing number of reviews that lack substance and are almost entirely made of filler content. Here’s an excerpt from a particularly egregious recent example I ran into, which had this to say on the subject of meaningful comparison to recent work: > Additionally, while the bibliography appears to be comprehensive, there could be some minor improvements, such as including more recent or relevant references if applicable. The whole review was written like this, with no specific suggestions for improvement, just vague “if applicable” filler. Infuriating.
Original comment Delving into ChatGPT usage in academic writing through excess vocabulary
thesz
Neutral mention · rule
> You can always prompt it to respond differently though. No, you cannot. I rarely do anything with LMs [1], but if I do I ask them to code "blocked clause decomposition" [2] in Haskell. Blocked clause decomposition is very, very simple as far as algorithm's implementation complexity goes and Haskell is just an unusual but simple implementation language. [1] https://arxiv.org/pdf/2403.07183.pdf [2] https://www.cs.utexas.edu/~marijn/publications/bcd.pdf I tried that on one less known model (my son sent me a link) and on Phind two times. The Phind's responces were very illuminating ones. On a first try, it tried to make me code that myself by pointing to a paper that does…
Read the full comment
not contain any blocked clause decomposition algorithm(s), just worthless combinations of them and references to other papers. Of course, I read that paper before. When I specifically asked to provide me with code, it did not even provided me with the correct type of blocked clause, which should be a pair of a CNF clause and a literal that blocks it. On a second try I could not even pass through bullshitting phase - I just was not able to persuade a LM to write any code. The second try I attempted after I saw an advert here on HN about Phind's new and updated LM, better at coding. Try that youself with your unusual but known problem of choice. And this is exactly a problem of collapse of knowledge: you can't ask an LM to code you a problem it hasn't seen several dozen of times during training. You just cannot, you will not get a satisfying answer.
Original comment AI and the Problem of Knowledge Collapse
myfonj
Neutral mention · rule
:] By their tongue, you will know them. Tangentially related: I really loved the 'shibboleth' that went like complex and multifaceted [things] that encompass [other things] from November 2023 [1]. Even though it was clear this will soon disappear from newer models being too obvious, I was still quite surprised, that "multifaceted" didn't even get into more recent study from March 2024 [2] that pointed out skyrocketing of terms such as commendable, intricate, and meticulous. [1] https://news.ycombinator.com/item?id=38501589 / https://blog.j11y.io/2023-11-22_multifaceted/ / https://twitter.com/padolsey/status/1727555475573440996 [2] https://news.ycombinator.com/item?id=39909692 / https://arxiv.org/abs/2403.07183 / https://twitter.com/myfonj/status/1773371415149891871
Original comment Google Books Is Indexing AI-Generated Garbage
a_bonobo
Neutral mention · rule
Worry not! People also use ChatGPT for peer review now. https://arxiv.org/abs/2403.07183 The entire scientific publication process is garbage. I say that as someone on ~140 papers who has quit academia: we focus on writing papers that nobody reads to tick boxes for career progression. The only one who benefits is the for-profit publishers who drown in taxpayer money. Why not automate the entire process?
Original comment AI-generated images got through peer review [video]

Keep following the thread.

More in Science & research

More from the same topics.

2018arXiv

Talent vs Luck: the role of randomness in success and failure

A. Pluchino. A. E. Biondo, A. Rapisarda

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.

Psychology
16 mentions14 months
2.5HN score