Abstract
Infrared-visibleimage fusion (IVIF) aims to generate a single, information-rich image for downstream tasks. However, prevailing methods exhibit two key limitations. First, many approaches lack explicit hierarchical semantic decoupling, failing to effectively integrate semantic features across different levels, which restricts their ability to capture complex scene structures. Second, task-driven fusion frameworks typically adopt a cascaded design, with unidirectional supervision provided by geometry-centric downstream tasks, such as detection. This architecture not only limits mutual reinforcement between the fusion and task networks, but also creates a 'supervision bottleneck' by lacking interaction with the linguistic modality that captures richer scene relationships. To tackle these challenges, we propose CCSFusion, the first framework to leverage chain-of-thought (CoT) captioning as supervision, redirecting IVIF optimization from narrow geometric accuracy to multimodal scene comprehension. It establishes a mutually reinforcing coupling between the fusion network and the captioning task. Specifically, we introduce a segmentation mask calibration unit (SMCU) to refine coarse semantic priors, providing precise pixel-level guidance. Subsequently, the calibrated features are fed into a chained semantic fusion module (CSFM), which explicitly decomposes the semantic priors into three hierarchical levels, and then feeds them into the hierarchical semantic attention (HSA) module. Finally, a bidirectional knowledge distillation mechanism transfers the reasoning ability of the teacher network to the student. Experiments show that CCSFusion achieves superior fusion performance and generates more semantically coherent images for high-level cognitive tasks. The code is available at: https://github.com/Snaillms/CCSFusion
| Original language | English |
|---|---|
| Pages (from-to) | 34830-34846 |
| Number of pages | 17 |
| Journal | IEEE Internet of Things Journal |
| Volume | 13 |
| Issue number | 15 |
| DOIs | |
| Publication status | Published - 1 Aug 2026 |
Keywords
- Chain-of-thought (CoT)
- hierarchical semantics
- image captioning
- infrared-visible image fusion (IVIF)
- task-driven fusion
Fingerprint
Dive into the research topics of 'CCSFusion: A Hierarchical Semantic Chain-of-Thought Reasoning Architecture for Infrared-Visible Image Fusion and Captioning'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver