Skip to content
Back to the collection

Nonreplicable publications are cited more than replicable ones

Marta Serra-Garcia, Uri Gneezy

2021 · Science Advances

Read the paper
HN score
0
Mentions
3
Readers
3
Months
3
What’s in a score?

From the abstract

Published papers that fail to replicate are cited more than those that replicate, even after the failure is published.

Metadata: Crossref. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

3 selected excerpts · 3 total mentions

araes
Neutral mention · rule
> spending time to carefully and deeply review a paper because they care and they feel on principal that’s the right thing to do Generally agree, although several parts of that issue. One of the first was covered by a paper back in 2023 that speaks to the issue about maximum extraction mode. [1] Fairness, honesty, and loyalty are usually rewarded with exploitation. If you spend time to carefully and deeply review the paper, then that ironically marks you as someone that can be exploited. You're implicitly marked as someone who will make personal sacrifices for the academic community and allow even more awful behavior to be piled on top of you. Unless they're caught with…
Read the full comment
something especially egregious, the people that don't, get promoted, spend less time on reviews, and get further rewards. [1] https://www.sciencedirect.com/science/article/abs/pii/S00221... The academic community has talked about this a bunch for years. Editors / reviewers that don't paid, or get minimal payment, and sacrifice large amounts of their personal time effectively volunteering, while authors pay $1000's for each paper submitted, and then journals charge $10,000's for each subscription. It's been talked about for decades, and yet in all that time, very little has actually occurred to change the situation. Another part on top of the "deeply reviewing papers" is that the sheer volume has massively increased (which has been an issue in a bunch of industries, sci-fi compilation Clarkesworld broke for quite a while in 2023 for similar reasons [2]). In the land of "type a sentence, and get a free academic paper" the extremely prolific are pouring out a paper a month, sometimes greater amounts. In areas like clinical medicine, hyper-prolific publishing has hit 70+ papers a year rates. [3] ~1.5 papers a week. Every few days somebody cranks out yet another paper that needs to be reviewed. In the article linked, one author had 140 articles to a single journal alone. Almost 3 times a week, all year long, you've got a paper claiming research worthy of publishing you need to review. [2] https://neil-clarke.com/how-ai-submissions-have-changed-our-... [3] https://www.sciencedirect.com/science/article/pii/S175115772... One that I have less direct, citeable proof for, yet am rather suspicious of, is that theft has also dramatically increased with a huge surge in invasive monitoring and snooping. If my TV changes what I'm watching, and what's recommended, because I typed a text message to somebody, it seems likely that a lot of academia is also dealing with massive intellectual theft issues. This then heavily prioritizes pouring out material as quickly as possible, with as little effort as possible, to get the equivalent of first post and maximal posts, before it can be scraped, exfiltrated, and published by somebody else. Finally, a lot of the reward and incentive has become metric chasing. Publish or Perish [4] and the Replication Crisis [5] are relatively well known ideas. Citation is a proxy of the impact of a paper, tenure and advancement is heavily related to quantity of publications and citations, and researchers would prefer to be cited more. And weirdly, if it does not work, and it's junk work, in a theme with the above, then it has been suggested nonreplicable publications are cited more than replicable ones [6]. In the linked paper, the view is that when "interesting" findings are published, they get more views, more media, more citations, and lower review standards get applied. And afterward there's very little social punishment for proving the results are false and not replicable (or reward for those illustrating lack of reproducability). Notably, the paper actually got a counterpoint stating that in psychology at least, lack of replication eventually predicts citation decline [7] (cited by 10), while the original actually got its authors ~250 citations, and a bunch of media mentions. [4] https://en.wikipedia.org/wiki/Publish_or_perish [5] https://en.wikipedia.org/wiki/Replication_crisis [6] https://www.science.org/doi/10.1126/sciadv.abd1705 [7] https://www.pnas.org/doi/10.1073/pnas.2304862120
tomjakubowski
Neutral mention · rule
Nonreplicable publications are cited more than replicable ones (2021) > We use publicly available data to show that published papers in top psychology, economics, and general interest journals that fail to replicate are cited more than those that replicate. This difference in citation does not change after the publication of the failure to replicate. Only 12% of postreplication citations of nonreplicable findings acknowledge the replication failure. https://www.science.org/doi/10.1126/sciadv.abd1705 Press release: https://rady.ucsd.edu/why/news/2021/05-21-a-new-replication-...

Keep following the thread.

More in Science & research

More from the same topics.