M³ SegNet: A Multi-Modal and Multi-Branch Framework for Nasopharyngeal Carcinoma Segmentation in Radiotherapy Planning

Accurate and simultaneous labeling of multiple structures, including gross tumor volumes, clinical target volumes, and organs at risk, is a fundamental multi-task requirement for radiotherapy planning in nasopharyngeal carcinoma. However, conventional manual labeling is labor-intensive and suffers from substantial inter-observer variability. This variability poses a significant challenge to the multi-modal interpretation of CT and MRI scans. Against this backdrop, automated approaches, particularly multi-modal and multi-task learning, are promising solutions. However, their clinical adoption is limited by three urgent needs: attention mechanisms that fuse multi-modal information at both local and global views, explicit incorporation of anatomical priors to regularize predictions, and a unified framework that enables concurrent segmentation of all desired structures. To overcome these limitations, we propose M³ SegNet, a novel multi-modal and multi-branch framework that concurrently performs all clinically relevant segmentation tasks, integrating feature fusion and anatomical guidance. Our primary contributions are threefold. First, we introduce the Synergistic Global-Local Attention that extracts informative features from various imaging modalities (CT, T1-weighted, T2-weighted, and T1 contrast). Second, we propose an Anatomy-Aware Hierarchical Learning strategy that uses OAR spatial information to guide tumor segmentation. We also integrate Random Modality Dropout to enhance robustness against missing modalities. We validated M³ SegNet on an internal 257-patient NPC dataset and confirmed its generalizability on three external datasets. In experiments, our framework significantly outperformed state-of-the-art methods. By providing a mechanism to leverage multi-modal information and anatomical priors, our M³ SegNet offers a reliable, automated, and clinically translatable solution for NPC radiotherapy planning.