TY - JOUR
T1 - DRFMamba
T2 - Multi-focus Microscopic Image Registration Fusion Based on Vision Mamba
AU - Song, Jie
AU - Xie, Xinyu
AU - Yang, Qiuhui
AU - Ng, Benjamin Koon Kei
AU - Ke, Wei
AU - Tong, Tong
AU - Sun, Yue
AU - Yu, Zitong
AU - Tan, Tao
N1 - Publisher Copyright:
© 1999-2012 IEEE.
PY - 2026
Y1 - 2026
N2 - Microscopic images provide detailed views of microstructures. They enable the observation and analysis of small biological samples, including their morphology and dynamics. However, these images often suffer from multi-focus and misalignment issues. To obtain clear and accurate images, registration and fusion are essential preprocessing steps. Currently, no unified network exists for multi-focus microscopic image registration and fusion, posing a significant challenge in extracting useful information from misaligned images. To address this challenge, we propose a high-performance model, Deep Registration Fusion Mamba (DRFMamba). The framework is divided into two stages: affine registration and fusion. In the first stage, to handle global rigid misalignment, we introduce the Deep Learning Vision Mamba Registration (DLVR) network for coarse affine registration of multi-focus microscopic images. In the DLVR network, a Linear Deformable Convolutional Vision Mamba (LDCVmamba) block is designed to capture both local and global displacement and deformation features from misaligned image pairs. This module enhances geometric disparity perception in in-focus regions while suppressing defocus interference, thereby improving affine registration accuracy. In the Deep Learning Vision Mamba Fusion (DLVF) network, a Laplacian Pyramid enhances multi-scale high-frequency representations, while the Vision Mamba (Vim) captures global structural features. Moreover, a Bidirectional Adaptive Cross-Alignment Fusion (ACAF) module achieves pixel-level feature alignment between image pairs, enabling bidirectional exchange of complementary information and adaptive fusion, further enhancing overall fusion performance. Extensive experimental results demonstrate that DRFMamba exhibits strong registration and fusion capabilities on multi-focus image datasets.
AB - Microscopic images provide detailed views of microstructures. They enable the observation and analysis of small biological samples, including their morphology and dynamics. However, these images often suffer from multi-focus and misalignment issues. To obtain clear and accurate images, registration and fusion are essential preprocessing steps. Currently, no unified network exists for multi-focus microscopic image registration and fusion, posing a significant challenge in extracting useful information from misaligned images. To address this challenge, we propose a high-performance model, Deep Registration Fusion Mamba (DRFMamba). The framework is divided into two stages: affine registration and fusion. In the first stage, to handle global rigid misalignment, we introduce the Deep Learning Vision Mamba Registration (DLVR) network for coarse affine registration of multi-focus microscopic images. In the DLVR network, a Linear Deformable Convolutional Vision Mamba (LDCVmamba) block is designed to capture both local and global displacement and deformation features from misaligned image pairs. This module enhances geometric disparity perception in in-focus regions while suppressing defocus interference, thereby improving affine registration accuracy. In the Deep Learning Vision Mamba Fusion (DLVF) network, a Laplacian Pyramid enhances multi-scale high-frequency representations, while the Vision Mamba (Vim) captures global structural features. Moreover, a Bidirectional Adaptive Cross-Alignment Fusion (ACAF) module achieves pixel-level feature alignment between image pairs, enabling bidirectional exchange of complementary information and adaptive fusion, further enhancing overall fusion performance. Extensive experimental results demonstrate that DRFMamba exhibits strong registration and fusion capabilities on multi-focus image datasets.
KW - Affine registration
KW - Cross-alignment fusion
KW - Multi-focus images
KW - Vision Mamba
UR - https://www.scopus.com/pages/publications/105041950370
U2 - 10.1109/TMM.2026.3702509
DO - 10.1109/TMM.2026.3702509
M3 - Article
AN - SCOPUS:105041950370
SN - 1520-9210
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
ER -