Infrared and visible image fusion via detail preserving adversarial learning

Jiayi Ma, Pengwei Liang, Wei Yu, Chen Chen, Xiaojie Guo, Jia Wu, Junjun Jiang*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

112 Citations (Scopus)

Abstract

TargefTablets can be detected easily from the background of infrared images due to their significantly discriminative thermal radiations, while visible images contain textural details with high spatial resolution which are beneficial to the enhancement of target recognition. Therefore, fused images with abundant detail information and effective target areas are desirable. In this paper, we propose an end-to-end model for infrared and visible image fusion based on detail preserving adversarial learning. It is able to overcome the limitations of the manual and complicated design of activity-level measurement and fusion rules in traditional fusion methods. Considering the specific information of infrared and visible images, we design two loss functions including the detail loss and target edge-enhancement loss to improve the quality of detail information and sharpen the edge of infrared targets under the framework of generative adversarial network. Our approach enables the fused image to simultaneously retain the thermal radiation with sharpening infrared target boundaries in the infrared image and the abundant textural details in the visible image. Experiments conducted on publicly available datasets demonstrate the superiority of our strategy over the state-of-the-art methods in both objective metrics and visual impressions. In particular, our results look like enhanced infrared images with clearly highlighted and edge-sharpened targets as well as abundant detail information.

Original languageEnglish
Pages (from-to)85-98
Number of pages14
JournalInformation Fusion
Volume54
DOIs
Publication statusPublished - Feb 2020

Keywords

  • Convolution neural network
  • Detail preserving
  • Generative adversarial network
  • Image fusion
  • Infrared

Fingerprint

Dive into the research topics of 'Infrared and visible image fusion via detail preserving adversarial learning'. Together they form a unique fingerprint.

Cite this