Blurry Face Image Reconstruction Based on Asymmetric Kernel Convolution Combined with Semantic Confidence Embedding
HU Zhengping1,2, PAN Peiyun1, ZHENG Saiyue1, ZHAO Mengyao1, BI Shuai1, LIU Yang3
1. School of Information Science and Engineering, Yanshan University, Qinhuangdao 066004; 2. Hebei Key Laboratory of Information Transmission and Signal Processing, Yanshan University, Qinhuangdao 066004
Abstract Inspired by the principle of heterogeneous convolution, a blurry face image reconstruction algorithm based on asymmetric kernel convolution combined with semantic confidence embedding is proposed under the framework of deep learning. Aiming at the deficiency of symmetrical square convolution kernel in expressing important features during feature extraction,asymmetric kernel is employed to replace the symmetric square convolution kernel to enhance the feature expression ability of the square convolution kernel. In the reconstruction stage, the asymmetric kernel convolution is combined with the semantic confidence network to further extract the most benificial features of each type of semantic information for the results in the reconstruction. The confidence is combined to guide the network to train in a more suitable direction for reconstruction. Experimental results on CelebA and Helen datasets show that the proposed algorithm produces better reconstruction results.
Fund:General Program of National Natural Science Foundation of China(No. 61771420), Natural Science Foundation of Hebei Province(No. F2016203422)
Corresponding Authors:
HU Zhengping, Ph. D., professor. His research interests include image/video processing.
About author:: PAN Peiyun, master student. Her research interests include blurry image and low resolution recognition image reconstruction.ZHENG Saiyue, master student. Her research interests include pattern recognition.ZHAO Mengyao, Ph.D.candidate. Her research interests include video anomaly detection.BI Shuai, Ph.D.candidate. His research interests include video picture processing.LIU Yang, master student,senior engineer. His research interests include pattern recognition and knowledge graph application.
HU Zhengping,PAN Peiyun,ZHENG Saiyue等. Blurry Face Image Reconstruction Based on Asymmetric Kernel Convolution Combined with Semantic Confidence Embedding[J]. , 2021, 34(7): 646-654.
[1] BAI Y C, JIA H Z, JIANG M, et al. Single-Image Blind Deblurring Using Multi-scale Latent Structure Prior. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(7): 2033-2045. [2] PAN J S, REN W Q, HU Z, et al. Learning to Deblur Images with Exemplars. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(6): 1412-1425. [3] XU L, ZHENG S C, JIA J Y. Unnatural L0 Sparse Representation for Natural Image Deblurring//Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 1107-1114. [4] REN W Q, CAO X C, PAN J S, et al. Image Deblurring via Enhanced Low-Rank Prior. IEEE Transactions on Image Processing, 2016, 25(7): 3426-3437. [5] PAN J S, SUN D Q, PFISTER H, et al. Deblurring Images via Dark Channel Prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(10): 2315-2328. [6] YAN Y Y, REN W Q, GUO Y F, et al. Image Deblurring via Extreme Channels Prior//Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 4003-4011. [7] PAN J S, HU Z, SU Z X, et al. Deblurring Face Images with Exemplars//Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 47-62. [8] 杨 平,赵燕伟,郑建炜,等.图像组转置训练及非凸约束的去噪去模糊算法.模式识别与人工智能, 2019, 32(10): 917-926. (YANG P, ZHAO Y W, ZHENG J W, et al. Image Patch Transform Training and Non-convex Regularization for Image Denoising and Deblurring. Pattern Recognition and Artificial Intelligence, 2019, 32(10): 917-926.) [9] ZHONG L, CHO S, METAXAS D, et al. Handling Noise in Single Image Deblurring Using Directional Filters//Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 612-619. [10] 高青青,赵建伟,周正华.基于递归多尺度卷积网络的图像超分辨率重建.模式识别与人工智能, 2020, 33(11): 972-980. (GAO Q Q, ZHAO J W, ZHOU Z H. Image Super-Resolution Reconstruction Based on Recursive Multi-scale Convolutional Networks. Pattern Recognition and Artificial Intelligence, 2020, 33(11): 972-980.) [11] LU B Y, CHEN J C, CHELLAPPA R. UID-GAN: Unsupervised Image Deblurring via Disentangled Representations. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2020, 2(1): 26-39. [12] TAO X, GAO H Y, SHEN X Y, et al. Scale-Recurrent Network for Deep Image Deblurring//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 8174-8182. [13] LU B Y, CHEN J C, CHELLAPPA R. Unsupervised Domain-SpecificDeblurring via DisentangledRepresentations//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 10217-10226. [14] SHEN Z Y, LAI W S, XU T F, et al. Deep Semantic Face Deblurring//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 8260-8269. [15] SHEN Z Y, LAI W S, XU T F, et al. Exploiting Semantics for Face Image Deblurring. International Journal of Computer Vision, 2020, 128(7): 1829-1846. [16] YASARLA R, PERAZZI F, PATEL V M. Deblurring Face Images Using Uncertainty Guided Multi-stream Semantic Networks. IEEE Transactions on Image Processing, 2020, 29: 6251-6263. [17] DING X H, GUO Y C, DING G G, et al. ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks//Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 1911-1920. [18] LI Y Q, LI X K, XIAO C J, et al. EACNet: Enhanced Asymme-tric Convolution for Real-Time Semantic Segmentation. IEEE Signal Processing Letters, 2021, 28: 234-238. [19] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition//Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016:770-778. [20] VASWANI A, SHAZEER N, PARMAR N, et.al. Attention Is All You Need//Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2017: 6000-6010. [21] PARKHI O M, VEDALDI A, ZISSERMAN A. Deep Face Recognition//Proc of the British Machine Vision Conference. Guildford, UK: BMVA Press, 2015: 41.1-41.12. [22] PERDANA A B, PRAHARA A. Face Recognition Using Light-Convolutional Neural Networks Based on Modified VGG16 Model//Proc of the International Conference on Computer Science and Information Technology. Washington, USA: IEEE, 2019. DOI: 10.1109/ICoSNIKOM48755.2019.9111481.
2021, Vol. 34 
