QWNet: A quaternion wavelet network for spatial-frequency aware multi-modal image fusion

Multi-modal Image Fusion (MMIF) enhances visual tasks by combining the strengths of different image modalities to improve object visibility and texture details. However, existing methods face two major challenges: First, a lack of intrinsic frequency-domain awareness, relying heavily on complex filters and fusion techniques that can be less adaptive. Second, simplistic channel combination that overlooks essential complex inter-channel relationships. To address these issues, we propose QWNet, a novel Quaternion Wavelet Network that harnesses both spatial and frequency information to enhance the network's inductive bias towards local features. By integrating wavelet transforms, we decompose input modalities into high- and low-frequency components, capturing global structures and fine details. These components are represented as quaternions, enabling the network to model complex inter-channel dependencies often missed by traditional real-valued networks. We also introduce a Bidirectional Adaptive Attention Module (BAAM) for effective multi-modal information interaction and difference enhancement, and a Quaternion Cross-modal Fusion Module (QCFM) to strengthen inter-channel relationships and effectively combine key features from different modalities. Extensive experiments confirm that our QWNet outperforms existing methods in fusion quality and downstream tasks like semantic segmentation, using only 4.27 K parameters and a computational cost of 0.30G FLOPs. The source code will be available at https://github.com/Mrzhans/QWNet.