TY - GEN
T1 - Multimodal Breast MRI Language-Image Pretraining (MLIP)
T2 - 1st Deep Breast Workshop on AI and Imaging for Diagnostic and Treatment Challenges in Breast Care, Deep-Breath 2024
AU - Rasoolzadeh, Nika
AU - Zhang, Tianyu
AU - Gao, Yuan
AU - van Dijk, Jarek M.
AU - Yang, Qiuhui
AU - Tan, Tao
AU - Mann, Ritse M.
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
PY - 2025
Y1 - 2025
N2 - Breast magnetic resonance imaging (MRI) is widely recognized for its high sensitivity in detecting breast cancer. However, interpreting breast MRI scans remains a complex, time-consuming, and resource-intensive task, even for experienced radiologists. To address these challenges, artificial intelligence-based methods are increasingly being employed. In this study, we developed a multimodal breast MRI language-image pre-training (MLIP) approach as an initial exploration of a breast MRI foundation model to aid in the interpretation of scans. Two types of inferences were used to evaluate MLIP’s performance. First, MLIP could retrieve corresponding MRI cases from a dataset based on a query, achieving an area under the receiver operating characteristic curve of 0.717 for suspicious and malignant cases, 0.640 for dense breasts, and 0.601 for low background parenchymal enhancement (BPE). Second, MLIP demonstrated the ability to predict the level of disease suspicion for a given MRI case. The results suggest that MLIP has the potential to serve as a foundation model for breast MRI interpretation. Future work will focus on expanding its capabilities through various downstream tasks and integrating additional models to enhance overall performance.
AB - Breast magnetic resonance imaging (MRI) is widely recognized for its high sensitivity in detecting breast cancer. However, interpreting breast MRI scans remains a complex, time-consuming, and resource-intensive task, even for experienced radiologists. To address these challenges, artificial intelligence-based methods are increasingly being employed. In this study, we developed a multimodal breast MRI language-image pre-training (MLIP) approach as an initial exploration of a breast MRI foundation model to aid in the interpretation of scans. Two types of inferences were used to evaluate MLIP’s performance. First, MLIP could retrieve corresponding MRI cases from a dataset based on a query, achieving an area under the receiver operating characteristic curve of 0.717 for suspicious and malignant cases, 0.640 for dense breasts, and 0.601 for low background parenchymal enhancement (BPE). Second, MLIP demonstrated the ability to predict the level of disease suspicion for a given MRI case. The results suggest that MLIP has the potential to serve as a foundation model for breast MRI interpretation. Future work will focus on expanding its capabilities through various downstream tasks and integrating additional models to enhance overall performance.
KW - Breast Imaging
KW - Contrastive Learning
KW - Foundation model
KW - MRI
KW - NLP
UR - http://www.scopus.com/inward/record.url?scp=85219167434&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-77789-9_5
DO - 10.1007/978-3-031-77789-9_5
M3 - Conference contribution
AN - SCOPUS:85219167434
SN - 9783031777882
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 42
EP - 53
BT - Artificial Intelligence and Imaging for Diagnostic and Treatment Challenges in Breast Care - 1st Deep Breast Workshop, Deep-Breath 2024, Held in Conjunction with MICCAI 2024, Proceedings
A2 - Mann, Ritse M.
A2 - Zhang, Tianyu
A2 - Han, Luyi
A2 - Litjens, Geert
A2 - Tan, Tao
A2 - Truhn, Danial
A2 - Li, Shuo
A2 - Gao, Yuan
A2 - Doyle, Shannon
A2 - Martí Marly, Robert
A2 - Kather, Jakob Nikolas
A2 - Pinker-Domenig, Katja
A2 - Wu, Shandong
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 10 October 2024 through 10 October 2024
ER -
