Joint segmentation of retinal layers and fluid lesions in optical coherence tomography with cross-dataset learning

Background and objectives: Age-related macular degeneration (AMD) is the leading cause of irreversible vision loss among people over 50 years old, which manifests in the retina through various changes of retinal layers and pathological lesions. The accurate segmentation of optical coherence tomography (OCT) image features is crucial for the identification and tracking of AMD. Although the recent developments in deep neural network have brought profound progress in this area, accurately segmenting retinal layers and pathological lesions remains a challenging task because of the interaction between these two tasks. Methods: In this study, we propose a three-branch, hierarchical multi-task framework that enables joint segmentation of seven retinal layers and three types of pathological lesions. A regression guidance module is introduced to provide explicit shape guidance between sub-tasks. We also propose a cross-dataset learning strategy to leverage public datasets with partial labels. The proposed framework was evaluated on a clinical dataset consisting of 140 OCT B-scans with pixel-level annotations of seven retinal layers and three types of lesions. Additionally, we compared its performance with the state-of-the-art methods on two public datasets. Results: Comprehensive ablation showed that the proposed hierarchical architecture significantly improved performance for most retinal layers and pathological lesions, achieving the highest mean DSC of 76.88 %. The IRF also achieved the best performance with a DSC of 68.15 %. Comparative studies demonstrated that the hierarchical multi-task architecture could significantly enhance segmentation accuracy and outperform state-of-the-art methods. Conclusion: The proposed framework could also be generalized to other medical image segmentation tasks with interdependent relationships.