Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

ecology study from the 40s that was definitively debunked by a biology study in the 90s. Maybe it’s p-hacking, journals that focus entirely on publishing novel research over replication (and grant funders that act similarly), refusals to share data and methodological details/process transparency, etc that apply to all fields. [1] DOI: 10.1126/science.aac4716 [2] https://www.nature.com/articles/s41562-018-0399-z [3]10.1136/bmj.e2555 [4]10.1038/nrd3439-c1 [5]10.1038/s41562-024-02062-9

replicability of social science experiments in Nature and Science between 2010 and 2015: https://www.nature.com/articles/s41562-018-0399-z - Raise standards for preclinical cancer research: https://www.nature.com/articles/483531a - Reproducibility Project: Cancer Biology: https://www.cos.io/rpcb - A Survey on Data Reproducibility in Cancer Research Provides Insights into Our Limited Ability to Translate Findings from the Laboratory to the Clinic: https://journals.plos.org/plosone/article?id=10.1371/journal...

influenced by people who take whatever their biases are, create some experiment specifically designed to show them, and then play with the numbers or experiment's parameters until they manage to do so. This goes all the way back to (and certainly before) Zimbardo's Stanford Prison Experiment. There was nothing organic there. Participants, both prisoner and guard, were heavily coached on how to act and, in their own words, saw the experiment more as an acting role than emergent normal behavior. It seems to be that this perception is accurate. In a society where people are increasingly radicalizing on social views, we ought expect social sciences to become even more dysfunctional in the years to come. This sort of stuff is, in turn, casting a very dangerous cloud over the rest of science since people tend to extrapolate these actions and behaviors in the social sciences, to science as a whole. In my opinion we need to start creating a strong distinction between science driven by science that yields falsifiability, predictability, and is driven exclusively by direct experimental result -- as compared to not-quite-science that is based on models, abstract experimentation, is not falsifiable, and does not provide meaningful predictions. What I mean by meaningful is that the whole point of predictability is not to have something to encourage political action on as is often the case in social science, but to use as a litmus test for the accuracy of a hypothesis. If it's true then that provides strength to the hypothesis, if it's false then the hypothesis is false. Without falsifiability, predictions are worthless. [1] - https://www.bloomberg.com/view/articles/2018-08-30/predictio... [2] - https://www.nature.com/articles/s41562-018-0399-z