Image Super-Resolution Reconstruction Based on l1 Induced Lightweight Deep Networks
ZHANG Dabao1, ZHAO Jianwei1,2, ZHOU Zhenghua1
1. College of Sciences, China Jiliang University, Hangzhou 310018; 2. Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, Hangzhou 310018
Abstract Existing deep-learning based super-resolution reconstruction methods improve the reconstruction performance of networks by deepening networks. However, sharp increase of the number of network weights is caused by deepening networks, resulting in a huge burden for the storage and training network. With the consideration of the sparsity of noise, the cost of training network and the sharpness of reconstructed edges, an image super-resolution reconstruction is proposed based on l1 induced lightweight deep networks integrating with the idea of model-driven and data-driven. Firstly, the split Bregman algorithm and soft threshold operator are utilized to deduce an effective iterative algorithm from the l1 reconstruction optimization model with an edge regularization term. Secondly, a corresponding recursive deep network is designed for image reconstruction under the guidance of the iterative algorithm. Therefore, the proposed deep network is designed under the guidance of the reconstruction optimization model, and its derived recursive structure reduces the number of network weights due to its property of weight sharing. Experimental results show that the proposed method achieves good reconstruction performance with less number of network weights.
Fund:Natural Science Foundation of Zhejiang Province(No.LY22F020002,LSY19F020001)
Corresponding Authors:
ZHAO Jianwei, Ph.D., professor. Her research interests include deep learning and image processing.
About author:: ZHANG Dabao, master student. His research interests include deep learning and image processing.ZHOU Zhenghua, Ph.D., associate professor. His research interests include deep learning and image processing.
[1] YILDIRIM D, GÜNGÖR O. A Novel Image Fusion Method Using IKONOS Satellite Images. Journal of Geodesy and Geoinformation, 2012, 1(1): 27-34. [2] 李仲年,张涛,张道强.基于自监督边缘融合网络的MRI影像重建.模式识别与人工智能, 2021, 34(4): 361-366. (LI Z N, ZHANG T, ZHANG D Q. Self-Supervised Edge-Fusion Network for MRI Reconstruction. Pattern Recognition and Artificial Intelligence, 2021, 34(4): 361-366.) [3] ZHANG L, WU X L. An Edge-Guided Image Interpolation Algorithm via Directional Filtering and Data Fusion. IEEE Transactions on Image Processing, 2006, 15(8): 2226-2238. [4] ZHANG K B, GAO X B, TAO D C, et al. Single Image Super-Re-solution with Non-local Means and Steering Kernel Regression. IEEE Transactions on Image Processing, 2012, 21(11): 4544-4556. [5] FREEMAN W T, JONES T R, PASZTOR E C. Example-Based Super-Resolution. IEEE Computer Graphics and Applications, 2002, 22(2): 56-65. [6] DONG C, LOY C C, HE K M, et al. Learning a Deep Convolutional Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 184-199. [7] DONG C, LOY C C, TANG X O. Accelerating the Super-Resolution Convolutional Neural Network // Proc of the European Confe-rence on Computer Vision. Berlin, Germany: Springer, 2016: 391-407. [8] SHI W B, CABALLERO J, HUSZÁR F, et al. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1874-1883. [9] KIM J, LEE J K, LEE K M. Accurate Image Super-Resolution Using Very Deep Convolutional Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1646-1654. [10] LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 5835-5843. [11] ZHANG Y L, LI K P, LI K, et al. Image Super-Resolution Using Very Deep Residual Channel Attention Networks // Proc of the European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2018: 294-310. [12] ZHANG Y L, TIAN Y P, KONG Y, et al. Residual Dense Network for Image Super-Resolution // Proc of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 2472-2481. [13] LI J C, FANG F M, MEI K F, et al. Multi-scale Residual Network for Image Super-Resolution // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 527-542. [14] ZHANG M J, WU Q Q, ZHANG J, et al. Fluid Micelle Network for Image Super-Resolution Reconstruction. IEEE Transactions on Cybernetics, 2023, 53(1): 578-591. [15] ZHANG M J, WU Q Q, GUO J, et al. Heat Transfer-Inspired Network for Image Super-Resolution Reconstruction. IEEE Transactions on Neural Networks and Learning Systems, 2022. DOI: 10.1109/TNNLS.2022.3185529. [16] ZHANG M J, XIN J W, ZHANG J, et al. Curvature Consistent Network for Microscope Chip Image Super-Resolution. IEEE Tran-sactions on Neural Networks and Learning Systems, 2022. DOI: 10.1109/TNNLS.2022.3168540. [17] KIM J, LEE J K, LEE K M. Deeply-Recursive Convolutional Network for Image Super-Resolution // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1637-1645. [18] TAI Y, YANG J, LIU X M. Image Super-Resolution via Deep Recursive Residual Network // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 2790-2798. [19] REICHSTEIN M, CAMPS-VALLS G, STEVENS B, et al. Deep Learning and Process Understanding for Data-Driven Earth System Science. Nature, 2019, 566: 195-204. [20] REN C, HE X H, PU Y F, et al. Learning Image Profile Enhancement and Denoising Statistics Priors for Single-Image Super-Resolution. IEEE Transaction on Cybernetics, 2021, 51(7): 3535-3548. [21] DONG W S, WANG P Y, YIN W T, et al. Denoising Prior Driven Deep Neural Network for Image Restoration. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2019, 41(10): 2305-2318. [22] GOLDSTEIN T, OSHER S. The Split Bregman Method for L1-Re-gularized Problems. SIAM Journal on Imaging Sciences, 2009, 2(2): 323-343. [23] CHAN S H, WANG X R, ELGENDY O A. Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications. IEEE Transaction on Computational Imaging, 2017, 3(1): 84-98. [24] KINGMA D P, BA J L. Adam: A Method for Stochastic Optimization[C/OL]. [2022-11-07]. https://arxiv.org/pdf/1412.6980.pdf. [25] TIMOFTE R, AGUSTSSON E, VAN GOOL L, et al. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results // Proc of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2017: 1110-1121. [26] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-Complexity Single-Image Super-Resolution Based on Nonnegative Neighbor Embedding // Proc of the British Machine Vision Confe-rence. Guildford, UK: BMVA Press, 2012. DOI: 10.5244/C.26.135 [27] ZEYDE R, ELAD M, PROTTER M. On Single Image Scale-Up Using Sparse-Representations // Proc of the 7th International Conference on Curves and Surfaces. Berlin, Germany: Springer, 2010: 711-730. [28] ARBELÁEZ P, MAIRE M, FOWLKES C, et al. Contour Detection and Hierarchical Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 898-916. [29] HUANG J B, SINGH A, AHUJA N. Single Image Super-Resolution from Transformed Self-Exemplars // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 5197-5206. [30] TAI Y, YANG J, LIU X M, et al. MemNet: A Persistent Memory Network for Image Restoration // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 4549-4557. [31] LAN R S, SUN L, LIU Z B, et al. MADNet: A Fast and Lightweight Network for Single-Image Super Resolution. IEEE Transactions on Cybernetics, 2021, 51(3): 1443-1453. [32] TIMOFTE R, DE SMET V, VAN GOOL L. A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution // Proc of the Asian Conference on Computer Vision. Berlin, Germany: Springer, 2014: 111-126. [33] HUI Z, WANG X M, GAO X B. Fast and Accurate Single Image Super-Resolution via Information Distillation Network // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 723-731. [34] PANG S R, CHEN Z, YIN F L. Image Super-Resolution Based on Generalized Residual Network. Arabian Journal for Science and Engineering, 2022, 47(2): 1903-1920.
2022, Vol. 35 
