Combining artificial intelligence analysis with expert mammogram reading: Determining the optimal AI positivity cut-off point for the French population-based breast cancer screening program.
Authors
Affiliations (6)
Affiliations (6)
- CRCDC-IDF, Pôle Data Etude Recherche et Evaluation, 28 rue Desaix, Paris 75015, France. Electronic address: [email protected].
- CRCDC-IDF, Site du Val-de-Marne, Le banc de Sable, 7, Quai Gabriel Péri, Joinville-le-Pont 94340, France.
- CRCDC-IDF, 28 rue Desaix, Paris 75015, France.
- CRCDC-IDF, 28 rue Desaix, Paris 75015, France; Université Paris Saclay/Université de Versailles-Saint-Quentin-en-Yvelines (UVSQ) Département de Gastroentérologie, Versailles 78000, France; Assistance Publique des Hôpitaux de Paris (AP-HP), Hôpital Ambroise Paré, Service de Gastroentérologie, Boulogne Billancourt 92100, France.
- Université de Poitiers, Laboratoire de Mathématiques et Applications, DACTIM MIS Lab, I3M, CNRS UMR, Poitiers 86000, France; Incepto Medical, Paris 75008, France.
- CRCDC-IDF, 28 rue Desaix, Paris 75015, France; Université Sorbonne Paris Nord, Département de Radiologie, Villetaneuse 93430, France; Assistance Publique-Hôpitaux de Paris (AP-HP), Hôpital Jean-Verdier, Service Radiologie, Bondy 93146, France.
Abstract
Artificial intelligence (AI) is increasingly explored as a complement to radiologists in population-based breast cancer screening, yet optimal strategies for its integration remain unclear. The French program relies on systematic double reading, a model challenged by rising workload and radiologist shortages. This study evaluates the performance of an AI algorithm and examines its potential role within the existing workflow. A retrospective cohort study included 13,186 women aged 50-74 years who underwent screening mammography in Ile-de-France between 2018 and 2019. Mammograms classified as BI-RADS 1-2 at first reading were reviewed by a second radiologist and subsequently reinterpreted in 2023 using the Transpara© AI algorithm, which assigns a 0-100 risk score. ROC curve analysis identified an optimal positivity threshold (cut-off-P), and diagnostic performance metrics were calculated. Organizational scenarios considering AI placement upstream or downstream of the second reading were explored. The optimal cut-off-P was 36.2, yielding an AUC of 0.78. At this threshold, AI would have detected 18 of 22 cancers-including most interval-cancers- and reduced the absolute risk of interval-cancer after a negative screen by nine points. Applied upstream of the second reading, AI could have excluded approximately 60% of mammograms classified as benign at first reading, potentially reducing second-reader workload by two thirds, but at the cost of 4 missed cancers and 4807 false-positive classifications. However, false positives remained frequent, and a few cancers detected at second reading received low AI scores. AI shows substantial potential to streamline breast cancer screening by safely triaging negative examinations before second reading. Nonetheless, limitations related to false-positives and missed cancers support a complementary-rather than substitutive- role for AI, ideally positioned between first and second readings.