AI for radiographic COVID-19 detection selects shortcuts over signal

The best story I heard about machine learning and radiology was when folks were racing to try to detect COVID in lung X-rays. As I recall, one group had fairly good success, but eventually someone figured out that their data set had images from a low-COVID hospital and a high-COVID hospital, and the lettering on the images used different fonts. The ML model was detecting the font, not the COVID. [a bit of googling later...] Here's a link to what I think was the debunking study: https://www.nature.com/articles/s42256-021-00338-7 If you're not at a university, try searching for "AI for radiographic COVID-19 detection selects shortcuts over signal" and you'll probably be able to find an open-access copy.

Yea this is what I was thinking of. Another work in that direction: https://www.nature.com/articles/s42256-021-00338-7 I never heard of the tanks story, but I vaguely remember stuff in radiology which was a similar story basically, it learned some spurious correlation to the label.

Also, credulous people trusting the magic box too much. Towards the beginning of the pandemic there was a lot of this sort of stuff, say: https://www.nature.com/articles/s42256-021-00338-7