基于多残差网络的结构保持超分辨重建

doi:10.16451/j.cnki.issn1003-6059.202103005

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (4174 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要针对图像超分辨率重建中几何结构扭曲和细节缺失等问题,文中提出基于多残差网络的结构保持超分辨重建算法.在小波变换域和梯度域上进行深度学习.文中算法包含3种残差网络.残差梯度网络用于结构及边缘信息的重建.残差小波变换网络从整体上进行图像高频信息的重建.残差通道注意力网络通过调整网络注意力,着重学习重要的通道特征,从局部恢复图像高频信息,提高重建效率.实验表明,文中算法在定量结果和视觉效果方面均取得较优表现.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	张铭津
	彭晓琪
	郭杰
	李云松
	王楠楠
	高新波

关键词 ：超分辨率重建, 深度学习, 多残差网络, 结构保持

Abstract：Aiming at the problems of geometric structure distortion and missing details in image super-resolution reconstruction, a structure-preserving super-resolution reconstruction algorithm based on multi-residual network is proposed. Deep learning is carried out in the wavelet transform domain and the gradient domain. Three kinds of residual networks are introduced. The structure and the edge information are reconstructed by the residual gradient network. The high-frequency information of the image is reconstructed as a whole by the residual wavelet transform network. The network attention is adjusted by the residual channel attention network , the important channel features are emphatically learned, and the high frequency information of the image is recovered locally. Experiments show that the proposed algorithm achieves better performance in both quantitative results and visual effects.

Key words： Super-resolution Reconstruction Deep Learning Multi-residual Networks Structure Preservation

收稿日期: 2020-09-25

ZTFLH:

TP 391.4

基金资助:国家自然科学基金青年基金项目(No.61902293)、陕西省高校科协人才托举计划项目(No.20200103)、中国科学院光谱成像技术重点实验室开放基金项目(No.LSIT201901W)、中央基本科研业务费新教师创新项目(No.XJS200112)资助

通讯作者: 高新波,博士,教授,主要研究方向为模式识别、计算机视觉.E-mail:gaoxb@cqupt.edu.cn.

作者简介: 张铭津,博士,副教授,主要研究方向为模式识别、计算机视觉.E-mail:mjinzhang@xidian.edu.cn.彭晓琪,硕士研究生,主要研究方向为模式识别、计算机视觉.E-mail:xqpengxidian@163.com.郭杰,博士,副教授,主要研究方向为图像传输与处理.E-mail:jguo@mail.xidian.edu.cn.李云松,博士,教授,主要研究方向为图像传输与处理.E-mail:ysli@mail.xidian.edu.cn.王楠楠,博士,教授,主要研究方向为模式识别、计算机视觉.E-mail:nnwang@xidian.edu.cn.

引用本文:

张铭津, 彭晓琪, 郭杰, 李云松, 王楠楠, 高新波. 基于多残差网络的结构保持超分辨重建[J]. 模式识别与人工智能, 2021, 34(3): 232-240. ZHANG Mingjin, PENG Xiaoqi, GUO Jie, LI Yunsong, WANG Nannan, GAO Xinbo. Structure-Preserving Super-Resolution Reconstruction Based on Multi-residual Network. , 2021, 34(3): 232-240.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/10.16451/j.cnki.issn1003-6059.202103005 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2021/V34/I3/232

[1] 付利华,孙晓威,赵宇,等.基于运动特征融合的快速视频超分辨率重构方法.模式识别与人工智能, 2019, 32(11): 1022-1031.
(FU L H, SUN X W, ZHAO Y, et al. Fast Video Super-Resolution Reconstruction Method Based on Motion Feature Fusion. Pattern Recognition and Artificial Intelligence, 2019, 32(11): 1022-1031.
[2] LEDIG C, THEIS L, HUSZÁR F, et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 105-114.
[3] DONG C, CHEN C L, TANG X O. Accelerating the Super-Resolution Convolutional Neural Network // Proc of the 14th European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 391-407.
[4] 付利华,卢中山,孙晓威,等.融合参考图像的人脸超分辨率重构方法.模式识别与人工智能, 2020, 33(4): 325-336.
(FU L H, LU Z S, SUN X W, et al. Face Super-Resolution Reconstruction Method Fusing Reference Image. Pattern Recognition and Artificial Intelligence, 2020, 33(4): 325-336.)
[5] SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1874-1883.
[6] ZOU W W W, YUEN P C. Very Low Resolution Face Recognition Problem // Proc of the 4th IEEE International Conference on Biometrics: Theory, Applications and Systems. Washington, USA: IEEE, 2010. DOI: 10.1109/BTAS.2010.5634490.
[7] SHI W Z, CABALLERO J, LEDIG C, et al. Cardiac Image Super-Resolution with Global Correspondence Using Multi-atlas PatchMatch // Proc of the International Conference on Medical Image Computing and Computer Assisted Intervention. Berlin, Germany: Springer, 2013: 9-16.
[8] 郑少飞,汤进,罗斌,等.基于改进损失函数的多阶段行人属性识别方法.模式识别与人工智能, 2018, 31(12): 1085-1095.
(ZHENG S F, TANG J, LUO B, et al. Multistage Pedestrian Attribute Recognition Method Based on Improved Loss Function. Pattern Recognition and Artificial Intelligence, 2018, 31(12): 1085-1095.)
[9] 蒋桧慧,张荣,李小宝,等.融合直接度量和间接度量的行人再识别.模式识别与人工智能, 2018, 31(2): 167-174.
(JIANG H H, ZHANG R, LI X B, et al. Pedestrian Re-identification Fusing Direct Metric and Indirect Metric. Pattern Recognition and Artificial Intelligence, 2018, 31(2): 167-174.)
[10] 刘丽,郑洋,付冬梅.改进YOLOv3网络结构的遮挡行人检测算法.模式识别与人工智能, 2020, 33(6): 568-574.
(LIU L, ZHENG Y, FU D M, et al. Occluded Pedestrian Detection Algorithm Based on Improved Network Structure of YOLOv3. Pattern Recognition and Artificial Intelligence, 2020, 33(6): 568-574.)
[11] DONG C, LOY C C, HE K M, et al. Learning a Deep Convolutional Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 184-199.
[12] KIM J, LEE J K, LEE K M. Accurate Image Super-Resolution Using Very Deep Convolutional Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1646-1654.
[13] TAI Z, YANG J, LIU X M. Image Super-Resolution via Deep Recursive Residual Network // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 2790-2798.
[14] KIM J, LEE J K, LEE K M. Deeply-Recursive Convolutional Network for Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1637-1645.
[15] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778.
[16] LIM B, SON S, KIM M, et al. Enhanced Deep Residual Networks for Single Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 136-144.
[17] TAI Y, YANG J, LIU X M, et al. MemNet: A Persistent Memory Network for Image Restoration // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 4549-4557.
[18] CHOI J S, KIM M. A Deep Convolutional Neural Network with Selection Units for Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 1150-1156.
[19] LI J C, FANG F M, MEI K F, et al. Multi-scale Residual Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 527-542.
[20] AHN N, KONG B, SOHN K A. Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 252-268.
[21] HE X Y, MO Z T, WANG P S, et al. Ode-Inspired Network Design for Single Image Super-Resolution // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 1732-1741.
[22] ZHANG Y L, LI K P, LI K, et al. Image Super Resolution Using Very Deep Residual Channel Attention Networks // Proc of the European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2018: 294-310.
[23] LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 5835-5843.
[24] SUN J, XU Z B, SHUM H Y. Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement. IEEE Transactions on Image Processing, 2010, 20(6): 1529-1542.
[25] KUMAR N, VERMA R, SETHI A. Convolutional Neural Networks for Wavelet Domain Super Resolution. Pattern Recognition Letters, 2017, 90: 65-71.
[26] TIMOFTE R, AGUSTSSON E, VAN GOOL L, et al. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 1110-1121.
[27] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-Complexity Single-Image Super-Resolution Based on Nonnegative Neigh-bor Embedding // Proc of the British Machine Vision Conference. Berlin, Germany: Springer, 2012: 1-10.
[28] DE SOUZA FARIA G, KIM H Y. Differential Audio Analysis: A New Side-Channel Attack on Pin Pads. International Journal of Information Security, 2019, 18: 73-84.
[29] MARTIN D, FOWLKES C C, TAL D, et al. A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics // Proc of the International Conference on Computer Vision. Washington, USA: IEEE, 2001: 416-423.
[30] HUANG J B, SINGH A, AHUJA N. Single Image Super-Resolution from Transformed Self-exemplars // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 5197-5206.
[31] WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.