模式识别与人工智能
Saturday, March 15, 2025 Home      About Journal      Editorial Board      Instructions      Ethics Statement      Contact Us                   中文
  2021, Vol. 34 Issue (11): 1028-1037    DOI: 10.16451/j.cnki.issn1003-6059.202111006
Deep Learning Design and Application Current Issue| Next Issue| Archive| Adv Search |
End-to-End Infrared and Visible Image Fusion Method Based on GhostNet
CHENG Chunyang1, WU Xiaojun1, XU Tianyang1
1. Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence, School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122

Download: PDF (1948 KB)   HTML (1 KB) 
Export: BibTeX | EndNote (RIS)      
Abstract  

Most of the existing deep learning based infrared and visible image fusion methods are grounded on manual fusion strategies in the fusion layer. Therefore ,they are incapable of designing an appropriate fusion strategy for the specific image fusion task.To overcome this problem, an end-to-end infrared and visible image fusion method based on GhostNet is proposed.The Ghost module is employed to replace the ordinary convolution layer in the network architecture, and thus it becomes a lightweight model.The constraint of the loss function makes the network learn the adaptive image features for the fusion task, and consequently the feature extraction and fusion are accomplished at the same time. In addition, the perceptual loss is introduced into the design of the loss function. The deep semantic information of source images is utilized in the image fusion as well.Source images are concatenated in the channel dimension and then fed into the deep network.A densely connected encoder is applied to extract deep features of source images. The fusion result is obtained through the reconstruction of the decoder. Experiments show that the proposed method is superior in subjective comparison and objective image quality evaluation metrics.

Key wordsInfrared Image      Visible Image      Image Fusion      Deep Learning     
Received: 17 May 2021     
ZTFLH: TN 911.73  
Fund:

National Natural Science Foundation of China(No.62020106012,U1836218,61672265), the 111 Project of Ministry of Education of China(No.B12018)

Corresponding Authors: WU Xiaojun, Ph.D., professor. His research interests include artificial intelligence, pattern recognition and computer vision.   
About author:: Cheng Chunyang, master student. His research interests include image fusion and deep learning.
XU Tianyang, Ph.D., associate professor. His research interests include artificial intelligence, pattern recognition and computer vision.
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
CHENG Chunyang
WU Xiaojun
XU Tianyang
Cite this article:   
CHENG Chunyang,WU Xiaojun,XU Tianyang. End-to-End Infrared and Visible Image Fusion Method Based on GhostNet[J]. , 2021, 34(11): 1028-1037.
URL:  
http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.202111006      OR     http://manu46.magtech.com.cn/Jweb_prai/EN/Y2021/V34/I11/1028
Copyright © 2010 Editorial Office of Pattern Recognition and Artificial Intelligence
Address: No.350 Shushanhu Road, Hefei, Anhui Province, P.R. China Tel: 0551-65591176 Fax:0551-65591176 Email: bjb@iim.ac.cn
Supported by Beijing Magtech  Email:support@magtech.com.cn