Quaternion Cross-modality Spatial Learning for Multi-modal Medical Image Segmentation

Junyang Chen, Guoheng Huang, Xiaochen Yuan, Guo Zhong, Zewen Zheng, Chi Man Pun, Jian Zhu, Zhixin Huang

Research output: Contribution to journalArticlepeer-review

Abstract

Recently, the Deep Neural Networks (DNNs) have had a large impact on imaging process including medical image segmentation, and the real-valued convolution of DNN has been extensively utilized in multi-modal medical image segmentation to accurately segment lesions via learning data information. However, the weighted summation operation in such convolution limits the ability to maintain spatial dependence that is crucial for identifying different lesion distributions. In this paper, we propose a novel Quaternion Cross-modality Spatial Learning (Q-CSL) which explores the spatial information while considering the linkage between multi-modal images. Specifically, we introduce to quaternion to represent data and coordinates that contain spatial information. Additionally, we propose Quaternion Spatial-association Convolution to learn the spatial information. Subsequently, the proposed De-level Quaternion Cross-modality Fusion (De-QCF) module excavates inner space features and fuses cross-modality spatial dependency. Our experimental results demonstrate that our approach compared to the competitive methods perform well with only 0.01061M parameters and 9.95G FLOPs. Our code is available at <uri>https://github.com/cjyang123456/Q-CSL.git</uri>.

Original languageEnglish
Pages (from-to)1-12
Number of pages12
JournalIEEE Journal of Biomedical and Health Informatics
DOIs
Publication statusAccepted/In press - 2023

Keywords

  • Biomedical imaging
  • Convolution
  • Cross-modality
  • Feature extraction
  • Image segmentation
  • Lesions
  • Multi-modal medical image
  • Quaternion
  • Quaternions
  • Spatial dependency
  • Three-dimensional displays

Fingerprint

Dive into the research topics of 'Quaternion Cross-modality Spatial Learning for Multi-modal Medical Image Segmentation'. Together they form a unique fingerprint.

Cite this