Lightweight Inverse Separable Residual Information Distillation Network for Image Super-Resolution Reconstruction
ZHAO Xiangqiang1,2,3, LI Xiyao1, SONG Zhaoyang1
1. College of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou 730050; 2. Key Laboratory of Gansu Advanced Control for Industrial Processes, Lanzhou University of Technology, Lanzhou 730050; 3. National Experimental Teaching Center of Electrical and Control Engineering, Lanzhou University of Technology, Lanzhou 730050
Abstract The application of the deep learning-based image super-resolution reconstruction algorithm on mobile devices is limited, due to the sharp increase of parameters and high computational cost caused by performance requirement. To solve this problem, a lightweight inverse separable residual information distillation network for image super-resolution reconstruction is proposed in this paper. Firstly, a progressive separable distillation shuffle module is designed to extract multi-level features and in the meantime keep the model lightweight. Multiple feature extraction connections are employed to learn a more distinguishing feature representation, and thus the network acquires more useful information from distillation. Then, a contrast perception coordinate attention module is designed to fully leverage channel-aware and position-sensitive information, enhancing the feature selection capability. Finally, a progressive compensation residual connection is put forward to improve the utilization of shallow features and compensate for the texture detail features of the network. Experiments show that the proposed algorithm achieves a good balance between model complexity and reconstruction performance, yielding excellent subjective and objective quality in the reconstructed high-resolution images.
Fund:National Key Research and Development Program(No.2020YFB1713600), National Natural Science Foundation of China(No.61763029)
Corresponding Authors:
ZHAO Xiaoqiang , Ph.D., professor. His research interests include image processing, fault diagnosis and data mining.
About author:: LI Xiyao, master student. His research interests include image processing and deep learning.SONG Zhaoyang, Ph.D., lecturer. His research interests include image processing and deep learning
[1] LIU H Y, RUAN Z B, ZHAO P, et al. Video Super-Resolution Based on Deep Learning: A Comprehensive Survey. Artificial Inte-lligence Review, 2022, 55(8): 5981-6035. [2] 宣锴,王乾. 面向磁共振图像重建的k空间降采样优化. 模式识别与人工智能, 2021, 34(4): 367-374. (XUAN K, WANG Q. Optimizing k-Space Subsampling Pattern toward MRI Reconstruction. Pattern Recognition and Artificial Intelligence, 2021, 34(4): 367-374.) [3] 王建,宋晓宁. 融合多尺度特征的轻量级人脸检测算法. 模式识别与人工智能, 2022, 35(6): 507-515. (WANG J, SONG X N. A Lightweight Face Detection Algorithm with Multi-scale Features Fusion. Pattern Recognition and Artificial Intelligence, 2022, 35(6): 507-515.) [4] FERNANDEZ-BELTRAN R, LATORRE-CARMONA P, PLA F. Single-Frame Super-Resolution in Remote Sensing: A Practical Overview. International Journal of Remote Sensing, 2017, 38(1): 314-354. [5] 孙旭,李晓光,李嘉锋,等. 基于深度学习的图像超分辨率复原研究进展. 自动化学报, 2017, 43(5): 697-709. (SUN X, LI X G, LI J F, et al. Review on Deep Learning Based Image Super-Resolution Restoration Algorithms. Acta Automatica Sinica, 2017, 43(5): 697-709.) [6] ZHANG L, WU X L. An Edge-Guided Image Interpolation Algorithm via Directional Filtering and Data Fusion. IEEE Transactions on Image Processing, 2006, 15(8): 2226-2238. [7] YANG J C, WANG Z W, LIN Z, et al. Coupled Dictionary Trai-ning for Image Super-Resolution. IEEE Transactions on Image Processing, 2012, 21(8): 3467-3478. [8] ZHANG K B, GAO X B, TAO D C, et al. Single Image Super-Re-solution with Non-local Means and Steering Kernel Regression. IEEE Transactions on Image Processing, 2012, 21(11): 4544-4556. [9] FUKAMI K, FUKAGATA K, TAIRA K. Super-Resolution Reconstruction of Turbulent Flows with Machine Learning. Journal of Fluid Mechanics, 2019, 870: 106-120. [10] DONG C, LOY C C, HE K M, et al. Learning a Deep Convolutional Network for Image Super-Resolution // Proc of European Conference on Computer Vision. Berlin, Germany:Springer, 2014: 184-199. [11] DONG C, LOY C C, TANG X O. Accelerating the Super-Resolution Convolutional Neural Network // Proc of the 14th European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 391-407. [12] KIM J, LEE J K, LEE K M. Accurate Image Super-Resolution Using Very Deep Convolutional Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1646-1654. [13] LIM B, SON S, KIM H, et al. Enhanced Deep Residual Networks for Single Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 1132-1140. [14] LI J C, FANG F M, MEI K F, et al. Multi-scale Residual Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 527-542. [15] TAI Y, YANG J, LIU X M, et al. MemNet: A Persistent Memory Network for Image Restoration // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 4549-4557. [16] KIM J, LEE J K, LEE K M. Deeply-Recursive Convolutional Network for Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1637-1645. [17] TAI Y, YANG J, LIU X M. Image Super-Resolution via Deep Recursive Residual Network // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 2790-2798. [18] ZHU F Y, ZHAO Q J. Efficient Single Image Super-Resolution via Hybrid Residual Feature Learning with Compact Back-Projection Network // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 2453-2460. [19] AHN N, KANG B, SOHN K A. Fast, Accurate,Lightweight Super-Resolution with Cascading Residual Network // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 256-272. [20] HUI Z, WANG X M, GAO X B. Fast and Accurate Single Image Super-Resolution via Information Distillation Network // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 723-731. [21] HUI Z, GAO X B, YANG Y C, et al. Lightweight Image Super-Resolution with Information Multi-distillation Network // Proc of the 27th ACM International Conference on Multimedia. New York, USA: ACM, 2019: 2024-2032. [22] ZHANG X Y, ZHOU X Y, LIN M X, et al. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices // Proc of the IEEE/CVF Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2018: 6848-6856. [23] HU J, SHEN L, SUN G. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. [24] ZHANG Y L, LI K P, LI K, et al. Image Super-Resolution Using Very Deep Residual Channel Attention Networks // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 294-310. [25] LU E M, HU X X. Image Super-Resolution via Channel Attention and Spatial Attention. Applied Intelligence, 2022, 52(2): 2260-2268. [26] ZHAO H Y, KONG X T, HE J W, et al. Efficient Image Super-Resolution Using Pixel Attention // Proc of the European Confe-rence on Computer Vision. Berlin, Germany: Springer, 2020: 56-72. [27] ZHANG Y L, TIAN Y P, KONG Y, et al. Residual Dense Network for Image Super-Resolution // Proc of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 2472-2481. [28] LIU J, TANG J, WU G S. Residual Feature Distillation Network for Lightweight Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 41-55. [29] LIU J, ZHANG W J, TANG Y T, et al. Residual Feature Aggregation Network for Image Super-Resolution // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 2356-2365. [30] HOU Q B, ZHOU D Q, FENG J S. Coordinate Attention for Efficient Mobile Network Design // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 13713-13722. [31] TIMOFTE R, AGUSTSSON E, VAN GOOL L, et al. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results // Proc of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE 2017: 1110-1121. [32] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-Complexity Single-Image Super-Resolution Based on Nonnegative Nei-ghbor Embedding // Proc of the British Machine Vision Confe-rence. Bristol, UK: BMVA, 2012. DOI: 10.5244/C.26.135. [33] ZEYDE R, ELAD M, PROTTER M. On Single Image Scale-up Using Sparse-Representations // Proc of the International Confe-rence on Curves and Surfaces. Berlin, Germany: Springer, 2012: 711-730. [34] ARBELÁEZ P, MAIRE M, FOWLKES C, et al. Contour Detection and Hierarchical Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 898-916. [35] HUANG J B, SINGH A, AHUJA N. Single Image Super-Resolution from Transformed Self-Exemplars // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 5197-5206. [36] GAO X B, LU W, TAO D C, et al. Image Quality Assessment Based on Multiscale Geometric Analysis. IEEE Transactions on Image Processing, 2009, 18(7): 1409-1423. [37] WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Asse-ssment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. [38] LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 5835-5843. [39] WANG L G, DONG X Y, WANG Y Q, et al. Exploring Sparsity in Image Super-Resolution for Efficient Inference // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 4915-4924. [40] PENG C M, SHU P, HUANG X Y, et al. LCRCA: Image Super-Resolution Using Lightweight Concatenated Residual Channel Attention Networks. Applied Intelligence, 2022, 52: 10045-10059. [41] YANG X, LI H H, LI X C. Lightweight Image Super-Resolution with Feature Cheap Convolution and Attention Mechanism. Cluster Computing, 2022, 25(6): 3977-3992. [42] YANG W M, WANG W, ZHANG X C, et al. Lightweight Feature Fusion Network for Single Image Super-Resolution. IEEE Signal Processing Letters, 2019, 26(4): 538-542.
2023, Vol. 36 
