基于特征聚合和传播网络的图像超分辨率重建

doi:10.16451/j.cnki.issn1003-6059.202404002

摘要
图/表
参考文献(38)
相关文章 (15)

全文: PDF (1866 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要

基于深度学习的图像超分辨率重建通过网络加深提升图像重建性能,但复杂网络会导致参数量急剧增加,限制其在资源受限设备上的应用.针对此问题,文中提出基于特征聚合和传播网络的图像超分辨率重建方法,采用逐步提取融合特征的方式获取图像丰富的内部信息.首先,提出上下文交互注意力模块,使网络学习到特征图丰富的上下文信息,提高特征的利用率.然后,设计多维注意力增强模块,提高网络对关键特征的判别能力,分别在通道和空间两个维度提取高频信息.最后,提出特征聚合传播模块,有效聚合深层细节信息,去除冗余信息,并促进有效信息在网络中传播.在Set5、Set14、BSD100、Urban100等基准数据集上的测试实验表明,文中方法性能较优,重建后的图像细节纹理较清晰.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	薄阳瑜
	刘晓晶
	武永亮
	王学军

关键词 ：图像超分辨率重建, 卷积神经网络, 上下文交互注意力, 多维注意力, 特征聚合

Abstract：

Image super-resolution reconstruction based on deep learning improves the image reconstruction performance by deepening the network. However, its application on resource-limited devices is limited due to the sharp increase in the number of parameters caused by complex networks. To solve this problem, an image super-resolution reconstruction method based on feature aggregation and propagation network is proposed, enriching internal information of images by extracting and fusing features step by step. Firstly, a contextual interaction attention block is proposed to enable the network to learn the rich contextual information of feature maps as well as improve the utilization of features. Then, a multi-dimensional attention enhancement block is designed to improve the network's ability to discriminate the key features and extract high-frequency information in channel dimension and spatial dimension, respectively. Finally, a feature aggregation and propagation block is proposed to effectively aggregate deep detail information, remove redundant information and promote the propagation of effective information in the network. Experimental results on Set5,Set14,BSD100 and Urban100 datasets demonstrate the superiority of the proposed method with clearer details of reconstructed images.

Key words： Image Super-Resolution Reconstruction Convolutional Neural Network Contextual Interaction Attention Multi-dimensional Attention Feature Aggregation

收稿日期: 2023-11-13

ZTFLH:	TP 751
	TP 391.41
	TP 183

基金资助:

国家自然科学基金项目(No.62106157)、河北省自然科学基金项目(No.F2021210002)、山西省重点实验室开放基金项目(No.CICIP2022001)资助

通讯作者: 王学军,博士,讲师,主要研究方向为图像处理、稀疏学习等.E-mail:wangxuejun@stdu.edu.cn.

作者简介: 薄阳瑜,硕士研究生,主要研究方向为图像处理、深度学习等.E-mail:1202110001@stdu.edu.cn. 刘晓晶,硕士研究生,主要研究方向为图像处理、深度学习等.E-mail:liuxiaojing0510@163.com. 武永亮,博士,讲师,主要研究方向为机器学习、自然语言处理等.E-mail:wuyongliang@stdu.edu.cn.

引用本文:

薄阳瑜, 刘晓晶, 武永亮, 王学军. 基于特征聚合和传播网络的图像超分辨率重建[J]. 模式识别与人工智能, 2024, 37(4): 299-312. BO Yangyu, LIU Xiaojing, WU Yongliang, Wang Xuejun. Image Super-Resolution Reconstruction Based on Feature Aggregation and Propagation Network. Pattern Recognition and Artificial Intelligence, 2024, 37(4): 299-312.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/10.16451/j.cnki.issn1003-6059.202404002 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2024/V37/I4/299

[1] FREEMAN W T, PASZTOR E C. Learning Low-Level Vision // Proc of the 7th IEEE International Conference on Computer Vision. Washington, USA: IEEE, 1999. DOI: 10.1109/ICCV.1999.790414.
[2] GLASNER D, BAGON S, IRANI M. Super-Resolution from a Single Image // Proc of the 12th IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2009: 349-356.
[3] RHEE S, KANG M G. Discrete Cosine Transform Based Regula-rized High-Resolution Image Reconstruction Algorithm. Optical Engineering, 1999, 38(8): 1348-1356.
[4] ZOU W W W, YUEN P C. Very Low Resolution Face Recognition Problem. IEEE Transactions on Image Processing, 2012, 21(1): 327-340.
[5] ZHANG L, WU X L. An Edge-Guided Image Interpolation Algorithm via Directional Filtering and Data Fusion. IEEE Transactions on Image Processing, 2006, 15(8): 2226-2238.
[6] YANG J C, WANG Z W, LIN Z, et al. Coupled Dictionary Trai-ning for Image Super-Resolution. IEEE Transactions on Image Processing, 2012, 21(8): 3467-3478.
[7] 张大宝,赵建伟,周正华.基于l₁诱导轻量级深度网络的图像超分辨率重建.模式识别与人工智能, 2022, 35(12): 1101-1110.
(ZHANG D B, ZHAO J W, ZHOU Z H. Image Super-Resolution Reconstruction Based on l₁ Induced Lightweight Deep Networks. Pattern Recognition and Artificial Intelligence, 2022, 35(12): 1101-1110.)
[8] 张焯林,赵建伟,曹飞龙.构建带空洞卷积的深度神经网络重建高分辨率图像.模式识别与人工智能, 2019, 32(3): 259-267.
(ZHANG Z L, ZHAO J W, CAO F L. Building Deep Neural Networks with Dilated Convolutions to Reconstruct High-Resolution Image. Pattern Recognition and Artificial Intelligence, 2019, 32(3): 259-267.)
[9] DONG C, LOY C C, HE K M, et al. Learning a Deep Convolutio-nal Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 184-199.
[10] LIM B, SON S, KIM H, et al. Enhanced Deep Residual Networks for Single Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2017: 1132-1140.
[11] ZHANG Y L, LI K P, LI K,et al. Image Super-Resolution Using Very Deep Residual Channel Attention Networks // Proc of the European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2018: 294-310.
[12] KIM J, LEE J K, LEE K M. Deeply-Recursive Convolutional Network for Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1637-1645.
[13] HUI Z, WANG X M, GAO X B. Fast and Accurate Single Image Super-Resolution via Information Distillation Network // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 723-731.
[14] HUI Z, GAO X B, YANG Y C, et al. Lightweight Image Super-Resolution with Information Multi-distillation Network // Proc of the 27th ACM International Conference on Multimedia. New York, USA: ACM, 2019: 2024-2032.
[15] KONG F Y, LI M X, LIU S W, et al. Residual Local Feature Network for Efficient Super-Resolution // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 765-775.
[16] ZHAO H Y, KONG X T, HE J W, et al. Efficient Image Super-Resolution Using Pixel Attention // Proc of the European Confe-rence on Computer Vision. Berlin, Germany: Springer, 2020: 56-72.
[17] CHEN H Y, GU J J, ZHANG Z. Attention in Attention Network for Image Super-Resolution[C/OL]. [2023-10-09]. https://arxiv.org/abs/2104.09497.
[18] ZHANG M J, WU Q Q, GUO J, et al. Heat Transfer-Inspired Network for Image Super-Resolution Reconstruction. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(2): 1810-1820.
[19] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-Complexity Single-Image Super-Resolution Based on Nonnegative Neigh-bor Embedding[C/OL]. [2023-10-09]. http://people.rennes.inria.fr/Aline.Roumy/publi/12bmvc_abstract_Bevilacqua_lowComplexitySR.pdf.
[20] ZEYDE R, ELAD M, PROTTER M. On Single Image Scale-Up Using Sparse-Representations // Proc of the 7th International Conference on Curves and Surfaces. Berlin, Germany: Springer, 2012: 711-730.
[21] TIMOFTE R, AGUSTSSON E, VAN GOOL L, et al. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results // Proc of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2017: 1110-1121.
[22] MARTIN D, FOWLKES C, TAL D, et al. A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics // Proc of the 8th IEEE International Conference on Computer Vision. Wa-shington, USA: IEEE, 2001: 416-423.
[23] HUANG J B, SINGH A, AHUJA N. Single Image Super-Resolution from Transformed Self-Exemplars // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 5197-5206.
[24] WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[25] TONG T, LI G, LIU X J, et al. Image Super-Resolution Using Dense Skip Connections // Proc of the IEEE International Confe-rence on Computer Vision. Washington, USA: IEEE, 2017: 4809-4817.
[26] ZHU F Y, ZHAO Q J. Efficient Single Image Super-Resolution via Hybrid Residual Feature Learning with Compact Back-Projection Network // Proc of the IEEE/CVF International Conference on Computer Vision Workshops. Washington, USA: IEEE, 2019: 2453-2460.
[27] LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 5835-5843.
[28] AHN N, KANG B, SOHN K A. Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 256-272.
[29] WANG L G, DONG X Y, WANG Y Q, et al. Exploring Sparsity in Image Super-Resolution for Efficient Inference // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 4915-4924.
[30] YANG W M, WANG W, ZHANG X C, et al. Lightweight Feature Fusion Network for Single Image Super-Resolution. IEEE Signal Processing Letters, 2019, 26(4): 538-542.
[31] LI J C, FANG F M, MEI K F, et al. Multi-scale Residual Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 527-542.
[32] YANG X, LI H K, LI X C. Lightweight Image Super-Resolution with Feature Cheap Convolution and Attention Mechanism. Cluster Computing, 2022, 25(6): 3977-3992.
[33] PENG C M, SHU P, HUANG X Y, et al. LCRCA: Image Super-Resolution Using Lightweight Concatenated Residual Channel Attention Networks. Applied Intelligence, 2022, 52: 10045-10059.
[34] PARK K, SOH J W, CHO N I. A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution. IEEE Transactions on Multimedia, 2023, 25: 907-918.
[35] 周登文,王婉君,马钰,等.基于区域互补注意力和多维注意力的轻量级图像超分辨率网络.模式识别与人工智能, 2022, 35(7): 625-636.
(ZHOU D W, WANG W J, MA Y, et al. Lightweight Image Super-Resolution Network Based on Regional Complementary Attention and Multi-dimensional Attemtion. Pattern Recognition and Artificial Intelligence, 2022, 35(7): 625-636.)
[36] LI X Y, SHAO Z H, LI B C, et al. Residual Shuffle Attention Network for Image Super-Resolution. Machine Vision and Applications, 2023, 34(5). DOI: 10.1007/s00138-023-01436-9.
[37] ZHU K Y. GUO S H. REN B, et al. Lightweight Image Super-Resolution with Expectation-Maximization Attention Mechanism. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(3): 1273-1284.
[38] 赵小强,李希尧,宋昭漾.轻量化逆可分离残差信息蒸馏网络的图像超分辨率重建.模式识别与人工智能, 2023, 36(5): 419-432.
(ZHANG X Q, LI X Y, SONG Z Y. Lightweight Inverse Separable Residual Information Distillation Network for Image Super-Reso-lution Reconstruction. Pattern Recognition and Artificial Intelligence, 2023, 36(5): 419-432.)