Neutral mention · rule
It used to be em dashes. And those really obvious words like „delve into“ I remember there was even a story about those words. EDIT: https://arxiv.org/abs/2406.07016
Back to the collection
Delving into LLM-assisted writing in biomedical publications through excess vocabulary
Dmitry Kobak, Rita González-Márquez, Emőke-Ágnes Horvát, Jan Lause
2024 · arXiv
- HN score
- 0
- Mentions
- 3
- Readers
- 3
- Months
- 3
From the abstract
Large language models (LLMs) like ChatGPT can generate and revise text with human-level performance. These models come with clear limitations: they can produce inaccurate information, reinforce existing biases, and be easily misused. Yet, many scientists use them for their scholarly writing. But how wide-spread is such LLM usage in the academic literature? To answer this question for the field of biomedical research, we present an unbiased, large-scale approach: we study vocabulary changes in over 15 million biomedical abstracts from 2010--2024 indexed by PubMed, and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. This excess word analysis suggests that at least 13.5% of 2024 abstracts were processed with LLMs. This lower bound differed across disciplines, countries, and journals, reaching 40% for some subcorpora. We show that LLMs have had an unprecedented impact on scientific writing in biomedical research, surpassing the effect of major world events such as the Covid pandemic.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
3 selected excerpts · 3 total mentions
Neutral mention · rule
Although "eager" isn't called out, a recent study of academic publications shows that the use of LLMs can be measured through word frequency analysis [1], finding certain words are disproportionally represented: > We study vocabulary changes in 14 million PubMed abstracts from 2010–2024, and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. 1: https://arxiv.org/html/2406.07016v1
Neutral mention · rule
There is some circumstantial evidence that the word 'delves' implies that ChatGPT wrote it. Some evidence here: https://arxiv.org/html/2406.07016v2 >Less common words with strong excess usage included delves (=28.0), showcasing (=10.2) and underscores (=10.9), together with their grammatical inflections (Figure 2a). But of course, that isn't a smoking gun, it might just be coincidence. The linked arxiv paper itself uses 'delves', probably tongue-in-cheek :)
Keep following the thread.
More in Biology & medicineMore from the same topics.
2023arXiv
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
60 mentions14 months
3.5HN score
2020arXiv
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
83 mentions35 months
3HN score
2018arXiv
Dissolving the Fermi Paradox
Anders Sandberg, Eric Drexler, Toby Ord
The Fermi paradox is the conflict between an expectation of a high {\em ex ante} probability of intelligent life elsewhere in the universe and the apparently lifeless universe we in fact observe.
35 mentions24 months
3HN score
2025arXiv
Design Patterns for Securing LLM Agents against Prompt Injections
Luca Beurer-Kellner, Beat Buesser, Ana-Maria Creţu +11
As AI agents powered by Large Language Models (LLMs) become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge.
4 mentions4 months
2.5HN score