Skip to content
Back to the collection

Raise standards for preclinical cancer research

C. Glenn Begley, Lee M. Ellis

2012 · Nature

Read the paper
HN score
0
Mentions
11
Readers
8
Months
10
What’s in a score?

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

11 selected excerpts · 11 total mentions

mike_hearn
Neutral mention · rule
Are you sure the NIH is critical for the pharma industry? Pharma companies generally conclude that replication rates of academic grant funded medical research are so low as to not be worth bothering with. From Amgen: "Over the past decade, before pursuing a particular line of research, scientists (including C.G.B.) in the haematology and oncology department at the biotechnology firm Amgen in Thousand Oaks, California, tried to confirm published findings ... scientific findings were confirmed in only 6 (11%) cases. Even knowing the limitations of preclinical research, this was a shocking result." [1] and from Bayer: "To mitigate some of the risks of such investments…
Read the full comment
ultimately being wasted, most pharmaceutical companies run in-house target validation programmes. However, validation projects that were started in our company based on exciting published data have often resulted in disillusionment when key data could not be reproduced. only in ∼20–25% of the projects were the relevant published data completely in line with our in-house findings Surprisingly, even publications in prestigious journals or from several independent groups did not ensure reproducibility Our observations indicate that literature data on potential drug targets should be viewed with caution If the stream of research that came out of NIH-grant funded work was genuinely useful, venture capitalists would be falling over themselves to commercialize it. In reality [3] [Atlas Venture partner] Booth said that the “unspoken rule” among early stage VCs is that at least 50% of published studies, even those in top-tier academic journals, “can’t be repeated with the same conclusions by an industrial lab. Atlas now insists on external validation studies of a new company’s basic science as a precondition to further investment.” He went on to say that the NIH should have higher standards and do more to vet the data. That never happened. He also said that due to the low quality of NIH output, "there’s such a scarcity of venture firms willing to take on early work, you rarely find yourself having to jump in quickly." That was 15 years ago. There was no serious effort to improve, and VCs are still much more interested in computing startups than biotech. [1] https://www.nature.com/articles/483531a [2] https://www.nature.com/articles/nrd3439-c1 [3] https://link.springer.com/article/10.1038/scibx.2011.416#pre...
phone8675309
Neutral mention · rule
I can see that - replication being the backbone of science. However, there are areas of inquiry that most would consider science that also suffer from similar failure rates of replication - for example, biomedical research[1]. Would you consider the people working in, say, oncology research[2], not doing science because there are issues replicating their work? (This is not a gotcha - just trying to understand your thought process and boundaries because perhaps you've seen things I haven't or understand things that I don't) [1] - https://www.nature.com/articles/nrd3439-c1 [2] - https://www.nature.com/articles/483531a
Isinlor
Neutral mention · rule
Assuming that everything in this pdf is correct, the conclusion should be that the "The Reproducibility Project: Psychology" does not provide evidence one way or another. There may be replication crisis, but the study could not tell the difference between replication crisis and 100% reproducibility. But today there are a lot more studies that find issues with replicability: - Many Labs 2: Investigating Variation in Replicability Across Samples and Settings: https://journals.sagepub.com/doi/10.1177/2515245918810225 - Reproducible brain-wide association studies require thousands of individuals: https://www.nature.com/articles/s41586-022-04492-9 - Evaluating the…
Read the full comment
replicability of social science experiments in Nature and Science between 2010 and 2015: https://www.nature.com/articles/s41562-018-0399-z - Raise standards for preclinical cancer research: https://www.nature.com/articles/483531a - Reproducibility Project: Cancer Biology: https://www.cos.io/rpcb - A Survey on Data Reproducibility in Cancer Research Provides Insights into Our Limited Ability to Translate Findings from the Laboratory to the Clinic: https://journals.plos.org/plosone/article?id=10.1371/journal...
d0mine
Neutral mention · rule
Yes, a significant proportion of all scientific research is not reproducible e.g., "Over half of psychology studies fail reproducibility test" https://www.nature.com/articles/nature.2015.18248 Cancer research: "scientific findings were confirmed in only 6 (11%) cases." https://www.nature.com/articles/483531a
nonbel
Neutral mention · rule
>"Are you honestly saying that medicine (a branch of biology) is not a "hard science"? That epidemiology, or genetics, or oncology isn't?" These fields in particular are not even close to being hard, especially oncology which appears to (somehow) be even worse off than psychology.[1:3] I bet less than 1% of people with masters or greater in a bio field have used calculus for anything in the last 5 years. While of course the definition of "hard science" is arbitrary, here are some decent overviews of the differences between physics and bio: https://www.cell.com/cancer-cell/fulltext/S1535-6108(02)0013... https://meehl.dl.umn.edu/sites/g/files/pua1696/f/074theoryte... Refs: [1] https://www.nature.com/articles/483531a [2] https://www.the-scientist.com/news-opinion/effort-to-reprodu... [3]https://www.sciencemag.org/news/2017/01/rigorous-replication...
nonbel
Neutral mention · rule
I was mentioning this in another thread and came across the new info I hadn't heard about yet. Here are some previous articles about this: March 2012: https://www.nature.com/articles/483531a June 2015: http://www.sciencemag.org/content/348/6242/1411 Dec 2015: https://www.nature.com/news/cancer-reproducibility-project-s... Jan 2017: https://www.nature.com/news/cancer-reproducibility-project-r...
nonbel
Neutral mention · rule
Its just an excuse, you are talking about a field with 10% replication rate, and when they tried to start doing more replications it was too expensive to even figure out what generated the data so they had to drop 25% of the studies before even beginning. One guy even calling the literature "augean stables": https://www.nature.com/news/cancer-reproducibility-project-s... Every disease can be arbitrarily split or combined into multiple categories. It doesnt mean anything. Here is a cure for cancer: 1) Detect aneuploid cells that fail to show the initial steps of the response to extracellular apoptosis signals, and have been dividing more often than usual for that…
Read the full comment
tissue/environment (eg you have to take into account wound healing, etc). 2) Kill those cells. This is technological problem, not a "cancer is many diseases" problem. Every problem can manifest in many ways and have multiple solutions. EDIT: Ref for "10% replication rate": >"scientific findings were confirmed in only 6 (11%) cases." https://www.nature.com/articles/483531a More info in this (short) thread from a few years ago: https://news.ycombinator.com/item?id=10687879
nonbel
Neutral mention · rule
>"Is this you? Your Ph.D. student has just finished 5 years of spectacular work, the manuscript and figures go through 12 revisions, then your colleagues and friends provide input for the next 10 revisions, you submit the perfect “version 22” manuscript to a carefully targeted scientific journal appropriate for the manuscript, and then you wait. And wait. And wait. Months elapse" Funny stuff. In the end most of these papers are not reproducible anyway, so what is taking so long? http://www.nature.com/articles/483531a

Keep following the thread.

More in Biology & medicine

More from the same topics.