Hidden Technical Debt in Machine Learning Systems

The work Google’s done in NLP has blown me away. It seems like they’ve spearheaded massive improvements 5 or 6 years in a row where computer vision has seen only incremental improvements in that time. I also appreciate their approach to ML ethics where they actually wrestle with some hard problems and come up with some solid approaches that engineers and data scientists can use (in contrast to one pre-print I read where the author—a guy with really good credentials—dogmatically insisted “do not solve problems that would lead to harms.” I’m sure route optimization has led to people having accidents on roads they would not have taken otherwise, should society not have built…

Two cents as an MD-(CS)PhD student studying what I've heard referred to as "the last mile problem." My stack trace of investigation: - The model is good, we just need to get the doctors to trust the model. - The model is good, we need to figure out how to build an informed trust in the model (so as to avoid automation bias). - The model is good, we need informed trust, but we can't tackle the trust issue without first figuring out a realistic deployment scenario. - The model is good, we need informed trust, we need a realistic deployment scenario, but there are some infrastructural issues that make deployment incredibly difficult. After painstaking work with real-life EHR…