Skip to content
The methodOpen data
Back to the collection

Underspecification Presents Challenges for Credibility in Modern Machine Learning

Alexander D'Amour, Katherine Heller, Dan Moldovan

Show all 40 authors

Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, D. Sculley

2020 · arXiv

Biology & medicineMachine learningPsychology
Read the paper
HN score
0
Mentions
3
Readers
3
Months
3
What’s in a score?

From the abstract

ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many predictors with equivalently strong held-out performance in the training domain. Underspecification is common in modern ML pipelines, such as those based on deep learning. Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains. This ambiguity can lead to instability and poor model behavior in practice, and is a distinct failure mode from previously identified issues arising from structural mismatch between training and deployment domains. We show that this problem appears in a wide variety of practical ML pipelines, using examples from computer vision, medical imaging, natural language processing, clinical risk prediction based on electronic health records, and medical genomics. Our results show the need to explicitly account for underspecification in modeling pipelines that are intended for real-world deployment in any domain.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

3 selected excerpts · 3 total mentions

pms
Neutral mention · rule
Have you provided the visual novel as a prompt or just asked about it? ChatGPT does seem to know more about books than songs, maybe because they are more often described in texts, while songs not, not sure. I wished more parameters gave a generalizable solution, but research suggests that's not the case [1, 2, 3]. OpenAI won't tell you this, cause they have a product to sell. [1] http://arxiv.org/abs/2211.08411 [2] http://arxiv.org/abs/2011.03395 [3] http://arxiv.org/abs/2005.04345
Original comment ChatGPT Plugins
andy99
Neutral mention · rule
You've probably seen it, but a more recent, related paper (that I think has some of the same authors) about inherent features of modern ML that make models so fragile, even if they test OK: Underspecification Presents Challenges for Credibility in Modern Machine Learning, D'Amour et al., https://arxiv.org/abs/2011.03395
Original comment Andrew Ng says AI has a proof-of-concept-to-production gap
throwaway888abc
Neutral mention · rule
Here is the paper. "ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many predictors with equivalently strong held-out performance in the training domain. Underspecification is common in modern ML pipelines, such as those based on deep learning. Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains. This ambiguity can lead to instability and poor model…
Read the full comment
behavior in practice, and is a distinct failure mode from previously identified issues arising from structural mismatch between training and deployment domains. We show that this problem appears in a wide variety of practical ML pipelines, using examples from computer vision, medical imaging, natural language processing, clinical risk prediction based on electronic health records, and medical genomics. Our results show the need to explicitly account for underspecification in modeling pipelines that are intended for real-world deployment in any domain." https://arxiv.org/abs/2011.03395
Original comment Google Reveals Major Hidden Weakness in Machine Learning

Keep following the thread.

More in Biology & medicine

More from the same topics.

2018arXiv

Talent vs Luck: the role of randomness in success and failure

A. Pluchino. A. E. Biondo, A. Rapisarda

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.

Psychology
16 mentions14 months
2.5HN score