TY - GEN
T1 - An Explainable Deep Framework
T2 - 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
AU - Han, Luyi
AU - Zhang, Tianyu
AU - Huang, Yunzhi
AU - Dou, Haoran
AU - Wang, Xin
AU - Gao, Yuan
AU - Lu, Chunyao
AU - Tan, Tao
AU - Mann, Ritse
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
PY - 2023
Y1 - 2023
N2 - Multi-sequence MRI is valuable in clinical settings for reliable diagnosis and treatment prognosis, but some sequences may be unusable or missing for various reasons. To address this issue, MRI synthesis is a potential solution. Recent deep learning-based methods have achieved good performance in combining multiple available sequences for missing sequence synthesis. Despite their success, these methods lack the ability to quantify the contributions of different input sequences and estimate region-specific quality in generated images, making it hard to be practical. Hence, we propose an explainable task-specific synthesis network, which adapts weights automatically for specific sequence generation tasks and provides interpretability and reliability from two sides: (1) visualize and quantify the contribution of each input sequence in the fusion stage by a trainable task-specific weighted average module; (2) highlight the area the network tried to refine during synthesizing by a task-specific attention module. We conduct experiments on the BraTS2021 dataset of 1251 subjects, and results on arbitrary sequence synthesis indicate that the proposed method achieves better performance than the state-of-the-art methods. Our code is available at https://github.com/fiy2W/mri_seq2seq.
AB - Multi-sequence MRI is valuable in clinical settings for reliable diagnosis and treatment prognosis, but some sequences may be unusable or missing for various reasons. To address this issue, MRI synthesis is a potential solution. Recent deep learning-based methods have achieved good performance in combining multiple available sequences for missing sequence synthesis. Despite their success, these methods lack the ability to quantify the contributions of different input sequences and estimate region-specific quality in generated images, making it hard to be practical. Hence, we propose an explainable task-specific synthesis network, which adapts weights automatically for specific sequence generation tasks and provides interpretability and reliability from two sides: (1) visualize and quantify the contribution of each input sequence in the fusion stage by a trainable task-specific weighted average module; (2) highlight the area the network tried to refine during synthesizing by a task-specific attention module. We conduct experiments on the BraTS2021 dataset of 1251 subjects, and results on arbitrary sequence synthesis indicate that the proposed method achieves better performance than the state-of-the-art methods. Our code is available at https://github.com/fiy2W/mri_seq2seq.
KW - Explainable Synthesis
KW - Missing-Sequence MRI Synthesis
KW - Multi-Sequence Fusion
KW - Task-Specific Attention
UR - http://www.scopus.com/inward/record.url?scp=85174675257&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-43999-5_5
DO - 10.1007/978-3-031-43999-5_5
M3 - Conference contribution
AN - SCOPUS:85174675257
SN - 9783031439988
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 45
EP - 55
BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings
A2 - Greenspan, Hayit
A2 - Greenspan, Hayit
A2 - Madabhushi, Anant
A2 - Mousavi, Parvin
A2 - Salcudean, Septimiu
A2 - Duncan, James
A2 - Syeda-Mahmood, Tanveer
A2 - Taylor, Russell
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 8 October 2023 through 12 October 2023
ER -