The software being tested comes from Vara, a Germany-based startup that also led the study. The company’s AI is already used in more than a quarter of Germany’s breast cancer screening centers and was introduced earlier this year at a hospital in Mexico and one in Greece.
Vara’s team, with the help of radiologists from the University Hospital of Essen in Germany and the Memorial Sloan Kettering Cancer Center in New York, tested two approaches. In the first case, AI works alone to analyze mammograms. In the second case, the AI automatically differentiates between scans it thinks look normal and those that cause concern. The latter refers to a radiologist, who would review them before seeing the AI assessment. Then the AI would issue an alert if it detects cancer and the doctor doesn’t.
“In the proposed AI-driven process, nearly three-quarters of screening studies did not need to be reviewed by a radiologist, while overall accuracy improved.”
Charles Langlotz
To train the neural network, Vara fed the AI data from over 367,000 mammograms—including radiologist notes, original assessments, and information about whether the patient ultimately had cancer—to learn how to put these images into one of three bins: “ sure normal ,” “not sure” (in which no prediction is given) and “sure cancer”. The conclusions of both approaches were then compared with the decisions originally made by real radiologists on 82,851 mammograms obtained from screening centers that did not contribute to the scans used to train the AI.
Another approach – a doctor and AI working together – was 3.6% better at detecting breast cancer than a doctor working alone, and caused fewer false alarms. It did this while automatically discarding images it classified as reliably normal, which amounted to 63% of all mammograms. This intensive rationalization could reduce the workload of radiologists.
After screening for breast cancer, patients with normal scans are sent on their way, while abnormal or inconclusive scans trigger follow-up testing. But radiologists who review mammograms miss 1 in 8 cancers. Fatigue, overwork, and even the time of day affect how well radiologists can identify tumors as they review thousands of images. Signs that are visually subtle are also less likely to set off alarms, and dense breast tissue – found mostly in younger patients – makes it harder to see the signs of cancer.
Radiologists using AI in the real world are required by German law to look at every mammogram, at least glancing at the ones the AI calls good. The AI still helps them by pre-populating scan reports marked as normal, although the radiologist can always decline the AI’s call.
Thilo Töllner, a radiologist who runs a German breast cancer screening center, has been using the program for two years. He sometimes disagrees when AI classifies scans as reliably normal and manually fills out reports to reflect a different conclusion, but says “normal people are almost always normal.” Basically, “you just need to press enter.”
Mammograms flagged by the AI as equivocal or “sure cancer” are referred to a radiologist – but only after the doctor offers an initial, independent assessment.
