Image super-resolution reconstruction based on deep learning improves the image reconstruction performance by deepening the network. However, its application on resource-limited devices is limited due to the sharp increase in the number of parameters caused by complex networks. To solve this problem, an image super-resolution reconstruction method based on feature aggregation and propagation network is proposed, enriching internal information of images by extracting and fusing features step by step. Firstly, a contextual interaction attention block is proposed to enable the network to learn the rich contextual information of feature maps as well as improve the utilization of features. Then, a multi-dimensional attention enhancement block is designed to improve the network's ability to discriminate the key features and extract high-frequency information in channel dimension and spatial dimension, respectively. Finally, a feature aggregation and propagation block is proposed to effectively aggregate deep detail information, remove redundant information and promote the propagation of effective information in the network. Experimental results on Set5,Set14,BSD100 and Urban100 datasets demonstrate the superiority of the proposed method with clearer details of reconstructed images.
薄阳瑜, 刘晓晶, 武永亮, 王学军. 基于特征聚合和传播网络的图像超分辨率重建[J]. 模式识别与人工智能, 2024, 37(4): 299-312.
BO Yangyu, LIU Xiaojing, WU Yongliang, Wang Xuejun. Image Super-Resolution Reconstruction Based on Feature Aggregation and Propagation Network. Pattern Recognition and Artificial Intelligence, 2024, 37(4): 299-312.
[1] FREEMAN W T, PASZTOR E C. Learning Low-Level Vision // Proc of the 7th IEEE International Conference on Computer Vision. Washington, USA: IEEE, 1999. DOI: 10.1109/ICCV.1999.790414.
[2] GLASNER D, BAGON S, IRANI M. Super-Resolution from a Single Image // Proc of the 12th IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2009: 349-356.
[3] RHEE S, KANG M G. Discrete Cosine Transform Based Regula-rized High-Resolution Image Reconstruction Algorithm. Optical Engineering, 1999, 38(8): 1348-1356.
[4] ZOU W W W, YUEN P C. Very Low Resolution Face Recognition Problem. IEEE Transactions on Image Processing, 2012, 21(1): 327-340.
[5] ZHANG L, WU X L. An Edge-Guided Image Interpolation Algorithm via Directional Filtering and Data Fusion. IEEE Transactions on Image Processing, 2006, 15(8): 2226-2238.
[6] YANG J C, WANG Z W, LIN Z, et al. Coupled Dictionary Trai-ning for Image Super-Resolution. IEEE Transactions on Image Processing, 2012, 21(8): 3467-3478.
[7] 张大宝,赵建伟,周正华.基于l1诱导轻量级深度网络的图像超分辨率重建.模式识别与人工智能, 2022, 35(12): 1101-1110.
(ZHANG D B, ZHAO J W, ZHOU Z H. Image Super-Resolution Reconstruction Based on l1 Induced Lightweight Deep Networks. Pattern Recognition and Artificial Intelligence, 2022, 35(12): 1101-1110.)
[8] 张焯林,赵建伟,曹飞龙.构建带空洞卷积的深度神经网络重建高分辨率图像.模式识别与人工智能, 2019, 32(3): 259-267.
(ZHANG Z L, ZHAO J W, CAO F L. Building Deep Neural Networks with Dilated Convolutions to Reconstruct High-Resolution Image. Pattern Recognition and Artificial Intelligence, 2019, 32(3): 259-267.)
[9] DONG C, LOY C C, HE K M, et al. Learning a Deep Convolutio-nal Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 184-199.
[10] LIM B, SON S, KIM H, et al. Enhanced Deep Residual Networks for Single Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2017: 1132-1140.
[11] ZHANG Y L, LI K P, LI K,et al. Image Super-Resolution Using Very Deep Residual Channel Attention Networks // Proc of the European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2018: 294-310.
[12] KIM J, LEE J K, LEE K M. Deeply-Recursive Convolutional Network for Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1637-1645.
[13] HUI Z, WANG X M, GAO X B. Fast and Accurate Single Image Super-Resolution via Information Distillation Network // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 723-731.
[14] HUI Z, GAO X B, YANG Y C, et al. Lightweight Image Super-Resolution with Information Multi-distillation Network // Proc of the 27th ACM International Conference on Multimedia. New York, USA: ACM, 2019: 2024-2032.
[15] KONG F Y, LI M X, LIU S W, et al. Residual Local Feature Network for Efficient Super-Resolution // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 765-775.
[16] ZHAO H Y, KONG X T, HE J W, et al. Efficient Image Super-Resolution Using Pixel Attention // Proc of the European Confe-rence on Computer Vision. Berlin, Germany: Springer, 2020: 56-72.
[17] CHEN H Y, GU J J, ZHANG Z. Attention in Attention Network for Image Super-Resolution[C/OL]. [2023-10-09]. https://arxiv.org/abs/2104.09497.
[18] ZHANG M J, WU Q Q, GUO J, et al. Heat Transfer-Inspired Network for Image Super-Resolution Reconstruction. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(2): 1810-1820.
[19] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-Complexity Single-Image Super-Resolution Based on Nonnegative Neigh-bor Embedding[C/OL]. [2023-10-09]. http://people.rennes.inria.fr/Aline.Roumy/publi/12bmvc_abstract_Bevilacqua_lowComplexitySR.pdf.
[20] ZEYDE R, ELAD M, PROTTER M. On Single Image Scale-Up Using Sparse-Representations // Proc of the 7th International Conference on Curves and Surfaces. Berlin, Germany: Springer, 2012: 711-730.
[21] TIMOFTE R, AGUSTSSON E, VAN GOOL L, et al. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results // Proc of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2017: 1110-1121.
[22] MARTIN D, FOWLKES C, TAL D, et al. A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics // Proc of the 8th IEEE International Conference on Computer Vision. Wa-shington, USA: IEEE, 2001: 416-423.
[23] HUANG J B, SINGH A, AHUJA N. Single Image Super-Resolution from Transformed Self-Exemplars // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 5197-5206.
[24] WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[25] TONG T, LI G, LIU X J, et al. Image Super-Resolution Using Dense Skip Connections // Proc of the IEEE International Confe-rence on Computer Vision. Washington, USA: IEEE, 2017: 4809-4817.
[26] ZHU F Y, ZHAO Q J. Efficient Single Image Super-Resolution via Hybrid Residual Feature Learning with Compact Back-Projection Network // Proc of the IEEE/CVF International Conference on Computer Vision Workshops. Washington, USA: IEEE, 2019: 2453-2460.
[27] LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 5835-5843.
[28] AHN N, KANG B, SOHN K A. Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 256-272.
[29] WANG L G, DONG X Y, WANG Y Q, et al. Exploring Sparsity in Image Super-Resolution for Efficient Inference // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 4915-4924.
[30] YANG W M, WANG W, ZHANG X C, et al. Lightweight Feature Fusion Network for Single Image Super-Resolution. IEEE Signal Processing Letters, 2019, 26(4): 538-542.
[31] LI J C, FANG F M, MEI K F, et al. Multi-scale Residual Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 527-542.
[32] YANG X, LI H K, LI X C. Lightweight Image Super-Resolution with Feature Cheap Convolution and Attention Mechanism. Cluster Computing, 2022, 25(6): 3977-3992.
[33] PENG C M, SHU P, HUANG X Y, et al. LCRCA: Image Super-Resolution Using Lightweight Concatenated Residual Channel Attention Networks. Applied Intelligence, 2022, 52: 10045-10059.
[34] PARK K, SOH J W, CHO N I. A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution. IEEE Transactions on Multimedia, 2023, 25: 907-918.
[35] 周登文,王婉君,马钰,等.基于区域互补注意力和多维注意力的轻量级图像超分辨率网络.模式识别与人工智能, 2022, 35(7): 625-636.
(ZHOU D W, WANG W J, MA Y, et al. Lightweight Image Super-Resolution Network Based on Regional Complementary Attention and Multi-dimensional Attemtion. Pattern Recognition and Artificial Intelligence, 2022, 35(7): 625-636.)
[36] LI X Y, SHAO Z H, LI B C, et al. Residual Shuffle Attention Network for Image Super-Resolution. Machine Vision and Applications, 2023, 34(5). DOI: 10.1007/s00138-023-01436-9.
[37] ZHU K Y. GUO S H. REN B, et al. Lightweight Image Super-Resolution with Expectation-Maximization Attention Mechanism. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(3): 1273-1284.
[38] 赵小强,李希尧,宋昭漾.轻量化逆可分离残差信息蒸馏网络的图像超分辨率重建.模式识别与人工智能, 2023, 36(5): 419-432.
(ZHANG X Q, LI X Y, SONG Z Y. Lightweight Inverse Separable Residual Information Distillation Network for Image Super-Reso-lution Reconstruction. Pattern Recognition and Artificial Intelligence, 2023, 36(5): 419-432.)
2024, Vol. 37 
