基于特征聚合和传播网络的图像超分辨率重建

doi:10.16451/j.cnki.issn1003-6059.202404002

Abstract
Figure/Table
References (38)
Related Citation (15)

Download: PDF (1866 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract

Image super-resolution reconstruction based on deep learning improves the image reconstruction performance by deepening the network. However, its application on resource-limited devices is limited due to the sharp increase in the number of parameters caused by complex networks. To solve this problem, an image super-resolution reconstruction method based on feature aggregation and propagation network is proposed, enriching internal information of images by extracting and fusing features step by step. Firstly, a contextual interaction attention block is proposed to enable the network to learn the rich contextual information of feature maps as well as improve the utilization of features. Then, a multi-dimensional attention enhancement block is designed to improve the network's ability to discriminate the key features and extract high-frequency information in channel dimension and spatial dimension, respectively. Finally, a feature aggregation and propagation block is proposed to effectively aggregate deep detail information, remove redundant information and promote the propagation of effective information in the network. Experimental results on Set5,Set14,BSD100 and Urban100 datasets demonstrate the superiority of the proposed method with clearer details of reconstructed images.

Key words： Image Super-Resolution Reconstruction Convolutional Neural Network Contextual Interaction Attention Multi-dimensional Attention Feature Aggregation

Received: 13 November 2023

ZTFLH:	TP 751
	TP 391.41
	TP 183

Fund:

National Natural Science Foundation of China(No.62106157), Natural Science Foundation of Hebei Province(No.F2021210002), Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province(No.CICIP2022001)

Corresponding Authors: WANG Xuejun, Ph.D., lecturer. His research interests include image processing and sparse learning.

About author:: BO Yangyu, Master student. Her research interests include image processing and deep learning.LIU Xiaojing, Master student. Her research interests include image processing and deep learning. WU Yongliang, Ph.D., lecturer. His research interests include machine learning and natural language processing.

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	BO Yangyu
	LIU Xiaojing
	WU Yongliang
	Wang Xuejun

Cite this article:

BO Yangyu,LIU Xiaojing,WU Yongliang等. Image Super-Resolution Reconstruction Based on Feature Aggregation and Propagation Network[J]. Pattern Recognition and Artificial Intelligence, 2024, 37(4): 299-312.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.202404002 OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2024/V37/I4/299

[1] FREEMAN W T, PASZTOR E C. Learning Low-Level Vision // Proc of the 7th IEEE International Conference on Computer Vision. Washington, USA: IEEE, 1999. DOI: 10.1109/ICCV.1999.790414.
[2] GLASNER D, BAGON S, IRANI M. Super-Resolution from a Single Image // Proc of the 12th IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2009: 349-356.
[3] RHEE S, KANG M G. Discrete Cosine Transform Based Regula-rized High-Resolution Image Reconstruction Algorithm. Optical Engineering, 1999, 38(8): 1348-1356.
[4] ZOU W W W, YUEN P C. Very Low Resolution Face Recognition Problem. IEEE Transactions on Image Processing, 2012, 21(1): 327-340.
[5] ZHANG L, WU X L. An Edge-Guided Image Interpolation Algorithm via Directional Filtering and Data Fusion. IEEE Transactions on Image Processing, 2006, 15(8): 2226-2238.
[6] YANG J C, WANG Z W, LIN Z, et al. Coupled Dictionary Trai-ning for Image Super-Resolution. IEEE Transactions on Image Processing, 2012, 21(8): 3467-3478.
[7] 张大宝,赵建伟,周正华.基于l₁诱导轻量级深度网络的图像超分辨率重建.模式识别与人工智能, 2022, 35(12): 1101-1110.
(ZHANG D B, ZHAO J W, ZHOU Z H. Image Super-Resolution Reconstruction Based on l₁ Induced Lightweight Deep Networks. Pattern Recognition and Artificial Intelligence, 2022, 35(12): 1101-1110.)
[8] 张焯林,赵建伟,曹飞龙.构建带空洞卷积的深度神经网络重建高分辨率图像.模式识别与人工智能, 2019, 32(3): 259-267.
(ZHANG Z L, ZHAO J W, CAO F L. Building Deep Neural Networks with Dilated Convolutions to Reconstruct High-Resolution Image. Pattern Recognition and Artificial Intelligence, 2019, 32(3): 259-267.)
[9] DONG C, LOY C C, HE K M, et al. Learning a Deep Convolutio-nal Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 184-199.
[10] LIM B, SON S, KIM H, et al. Enhanced Deep Residual Networks for Single Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2017: 1132-1140.
[11] ZHANG Y L, LI K P, LI K,et al. Image Super-Resolution Using Very Deep Residual Channel Attention Networks // Proc of the European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2018: 294-310.
[12] KIM J, LEE J K, LEE K M. Deeply-Recursive Convolutional Network for Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1637-1645.
[13] HUI Z, WANG X M, GAO X B. Fast and Accurate Single Image Super-Resolution via Information Distillation Network // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 723-731.
[14] HUI Z, GAO X B, YANG Y C, et al. Lightweight Image Super-Resolution with Information Multi-distillation Network // Proc of the 27th ACM International Conference on Multimedia. New York, USA: ACM, 2019: 2024-2032.
[15] KONG F Y, LI M X, LIU S W, et al. Residual Local Feature Network for Efficient Super-Resolution // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 765-775.
[16] ZHAO H Y, KONG X T, HE J W, et al. Efficient Image Super-Resolution Using Pixel Attention // Proc of the European Confe-rence on Computer Vision. Berlin, Germany: Springer, 2020: 56-72.
[17] CHEN H Y, GU J J, ZHANG Z. Attention in Attention Network for Image Super-Resolution[C/OL]. [2023-10-09]. https://arxiv.org/abs/2104.09497.
[18] ZHANG M J, WU Q Q, GUO J, et al. Heat Transfer-Inspired Network for Image Super-Resolution Reconstruction. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(2): 1810-1820.
[19] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-Complexity Single-Image Super-Resolution Based on Nonnegative Neigh-bor Embedding[C/OL]. [2023-10-09]. http://people.rennes.inria.fr/Aline.Roumy/publi/12bmvc_abstract_Bevilacqua_lowComplexitySR.pdf.
[20] ZEYDE R, ELAD M, PROTTER M. On Single Image Scale-Up Using Sparse-Representations // Proc of the 7th International Conference on Curves and Surfaces. Berlin, Germany: Springer, 2012: 711-730.
[21] TIMOFTE R, AGUSTSSON E, VAN GOOL L, et al. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results // Proc of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2017: 1110-1121.
[22] MARTIN D, FOWLKES C, TAL D, et al. A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics // Proc of the 8th IEEE International Conference on Computer Vision. Wa-shington, USA: IEEE, 2001: 416-423.
[23] HUANG J B, SINGH A, AHUJA N. Single Image Super-Resolution from Transformed Self-Exemplars // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 5197-5206.
[24] WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[25] TONG T, LI G, LIU X J, et al. Image Super-Resolution Using Dense Skip Connections // Proc of the IEEE International Confe-rence on Computer Vision. Washington, USA: IEEE, 2017: 4809-4817.
[26] ZHU F Y, ZHAO Q J. Efficient Single Image Super-Resolution via Hybrid Residual Feature Learning with Compact Back-Projection Network // Proc of the IEEE/CVF International Conference on Computer Vision Workshops. Washington, USA: IEEE, 2019: 2453-2460.
[27] LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 5835-5843.
[28] AHN N, KANG B, SOHN K A. Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 256-272.
[29] WANG L G, DONG X Y, WANG Y Q, et al. Exploring Sparsity in Image Super-Resolution for Efficient Inference // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 4915-4924.
[30] YANG W M, WANG W, ZHANG X C, et al. Lightweight Feature Fusion Network for Single Image Super-Resolution. IEEE Signal Processing Letters, 2019, 26(4): 538-542.
[31] LI J C, FANG F M, MEI K F, et al. Multi-scale Residual Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 527-542.
[32] YANG X, LI H K, LI X C. Lightweight Image Super-Resolution with Feature Cheap Convolution and Attention Mechanism. Cluster Computing, 2022, 25(6): 3977-3992.
[33] PENG C M, SHU P, HUANG X Y, et al. LCRCA: Image Super-Resolution Using Lightweight Concatenated Residual Channel Attention Networks. Applied Intelligence, 2022, 52: 10045-10059.
[34] PARK K, SOH J W, CHO N I. A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution. IEEE Transactions on Multimedia, 2023, 25: 907-918.
[35] 周登文,王婉君,马钰,等.基于区域互补注意力和多维注意力的轻量级图像超分辨率网络.模式识别与人工智能, 2022, 35(7): 625-636.
(ZHOU D W, WANG W J, MA Y, et al. Lightweight Image Super-Resolution Network Based on Regional Complementary Attention and Multi-dimensional Attemtion. Pattern Recognition and Artificial Intelligence, 2022, 35(7): 625-636.)
[36] LI X Y, SHAO Z H, LI B C, et al. Residual Shuffle Attention Network for Image Super-Resolution. Machine Vision and Applications, 2023, 34(5). DOI: 10.1007/s00138-023-01436-9.
[37] ZHU K Y. GUO S H. REN B, et al. Lightweight Image Super-Resolution with Expectation-Maximization Attention Mechanism. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(3): 1273-1284.
[38] 赵小强,李希尧,宋昭漾.轻量化逆可分离残差信息蒸馏网络的图像超分辨率重建.模式识别与人工智能, 2023, 36(5): 419-432.
(ZHANG X Q, LI X Y, SONG Z Y. Lightweight Inverse Separable Residual Information Distillation Network for Image Super-Reso-lution Reconstruction. Pattern Recognition and Artificial Intelligence, 2023, 36(5): 419-432.)