梯度控制与鉴别特征引导的写实类人物肖像插画转换方法

doi:10.16451/j.cnki.issn1003-6059.202011001

Abstract
Figure/Table
References (0)
Related Citation (15)

Download: PDF (5332 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract

Without considering the preservation of facial recognition characteristics,the existing unsupervised image-to-image translation methods often suffer from face distortion and facial structure collapse.Consequently,it is difficult to identify the personal information after translation.To tackle this issue,gradient controlled and discriminative features guided image-to-image translation method towards realistic portrait illustrations is proposed based on cycle-consistent generative adversarial network(CycleGAN).Masked residual long connections are introduced by the proposed method to avoid reusing redundant features and image gradient information consistency is considered as a constraint to preserve facial recognition characteristics.In addition,a discriminative feature guided information-shared training mechanism is devised and thus generators are capable of capturing discriminative features of target images,like the discriminators.Moreover,patch-level discriminators are extended to multi-awareness discriminators to obtain more discriminative information.Experimental results show that the proposed method preserves facial recognition characteristics well in the translated illustrations and outperforms the existing unsupervised image-to-image translation methods in visual effect of illustrations.

Key words： Portrait Illustration Image-to-Image Translation Cycle-Consistent Generative Adversarial Network(CycleGAN) Image Gradient Consistency Discriminative Feature

Received: 13 August 2020

ZTFLH:

TP391.41

Corresponding Authors: YE Dongyi,Ph.D.,professor.His research interests include computational intelligence and data mining.

About author:: SHI Rongxiao,master student.His research interests include intelligent image processing;CHEN Zhaojiong,master,professor.Her research interests include intelligent image processing and computational intelligence.

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	SHI Rongxiao
	YE Dongyi
	CHEN Zhaojiong

Cite this article:

SHI Rongxiao,YE Dongyi,CHEN Zhaojiong. Gradient Controlled and Discriminative Features Guided Image-to-Image Translation Method Towards Realistic Portrait Illustrations[J]. , 2020, 33(11): 959-971.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.202011001 OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2020/V33/I11/959

[1] 沈典靖,胡昌格.浅谈人物肖像插画在现代设计中的特征与应用.明日风尚,2016(23):44,39.
(SHEN D J,HU C G.On the Characteristics and Application of Portrait Illustration in Modern Design.MING,2016(23):44,39.)
[2] WU R Z,GU X D,TAO X,et al.Landmark Assisted CycleGAN for Cartoon Face Generation[C/OL].[2020-07-01].https://arxiv.org/pdf/1907.01424v1.pdf.
[3] ISOLA P,ZHU J Y,ZHOU T H,et al.Image-to-Image Translation with Conditional Adversarial Networks//Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2017:5967-5976.
[4] LI Y H,CHEN X J,WU F,et al.LinesToFacePhoto:Face Photo Generation from Lines with Conditional Self-attention Generative Adversarial Networks//Proc of the 27th ACM International Conference on Multimedia.New York,USA:ACM,2019:2323-2331.
[5] 顾杨,陈昭炯,陈灿,等.基于CGAN 的中国山水画布局可调的仿真生成方法.模式识别与人工智能,2019,32(9):844-854.
(GU Y,CHEN Z J,CHEN C,et al.Layout Adjustable Simulated Generation Method for Chinese Landscape Paintings Based on CGAN.Pattern Recognition and Artificial Intelligence,2019,32(9):844-854.)
[6] 王楠楠,李洁,高新波.人脸画像合成研究的综述与对比分析.模式识别与人工智能,2018,31(1):37-48.
(WANG N N,LI J,GAO X B.A Review and Comparison Study on Face Sketch Synthesis.Pattern Recognition and Artificial Intelligence,2018,31(1):37-48.)
[7] 张玉倩,高方远,王楠楠.基于深度概率图模型的鲁棒人脸画像合成.模式识别与人工智能,2019,32(9):855-866.
(ZHANG Y Q,GAO F Y,WANG N N.Robust Face Sketch Synthesis Based on Deep Probabilistic Graphical Models.Pattern Re-cognition and Artificial Intelligence,2019,32(9):855-866.)
[8] YI R,LIU Y J,LAI Y K,et al.APDrawingGAN:Generating Artistic Portrait Drawings from Face Photos with Hierarchical GANs//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2019:10743-10752.
[9] ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks//Proc of the IEEE International Conference on Computer Vision.Washington,USA:IEEE,2017:2242-2251.
[10] LIU M Y,BREUEL T,KAUTZ J.Unsupervised Image-to-Image Translation Networks//VON LUXBURG U,GUYON I,BENGIO S,et al.,eds.Advances in Neural Information Processing Systems 30.Cambridge,USA:The MIT Press,2017:700-708.
[11] HUANG X,LIU M Y,BELONGIE S,et al.Multimodal Unsupervised Image-to-Image Translation//Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2018:172-189.
[12] TOMEI M,CORNIA M,BARALDI L,et al. Art2Real:Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translation//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2019:5849-5859.
[13] SU H,NIU J W,LIU X F,et al. Unpaired Photo-to-Manga Translation Based on the Methodology of Manga Drawing[C/OL].[2020-07-01].https://arxiv.org/pdf/2004.10634v1.pdf.
[14] GATYS L A,ECKER A S,BETHGE M.Image Style Transfer Using Convolutional Neural Networks//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:2414-2423.
[15] LIAO J,YAO Y,YUAN L,et al.Visual Attribute Transfer through Deep Image Analogy[C/OL].[2020-07-01].https://arxiv.org/pdf/1705.01088.pdf.
[16] CHEN Y,LAI Y K,LIU Y J.CartoonGAN:Generative Adversarial Networks for Photo Cartoonization//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2018:9465-9474.
[17] WANG X R,YU J Z.Learning to Cartoonize Using White-Box Cartoon Representations//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2020:8090-8099.
[18] ZHANG Y,DONG W M,DEUSSEN O,et al.Data-Driven Face Cartoon Stylization//Proc of the SIGGRAPH Asia 2014 Technical Briefs.Berlin,Germany:Springer,2014.DOI:10.1145/2669024.2669028.
[19] ROSIN P L,LAI Y K.Non-photorealistic Rendering of Portraits[C/OL].[2020-07-01].https://core.ac.uk/download/pdf/74219072.pdf.
[20] WANG T H,COLLOMOSSE J,HUNTER A,et al. Learnable Stroke Models for Example-Based Portrait Painting[C/OL].[2020-07-01].http://www.bmva.org/bmvc/2013/Papers/paper0036/paper0036.pdf.
[21] HUANG X,BELONGIE S.Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization//Proc of the IEEE International Conference on Computer Vision.Washington,USA:IEEE,2017:1510-1519.
[22] LIU M Y,HUANG X,MALLYA A,et al.Few-Shot Unsupervised Image-to-Image Translation//Proc of the IEEE/CVF International Conference on Computer Vision.Washington,USA:IEEE,2019:10551-10560.
[23] DEKEL T,GAN C,KRISHNAN D,et al. Sparse,Smart Contours to Represent and Edit Images//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2018:3511-3520.
[24] HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:770-778.
[25] PASZKE A,GROSS S,CHINTALA S,et al.Automatic Differentiation in PyTorch[C/OL].[2020-07-01].https://openreview.net/pdf?id=BJJsrmfCZ.
[26] YANG S,LUO P,LOY C C,et al.From Facial Parts Responses to Face Detection:A Deep Learning Approach//Proc of the IEEE International Conference on Computer Vision.Washington,USA:IEEE,2015:3676-3684.
[27] WANG Z,BOVIK A C,SHEIKH H R,et al.Image Quality Assessment:From Error Visibility to Structural Similarity.IEEE Transactions on Image Processing,2004,13(4):600-612.
[28] HASLER D,SUESSTRUNK S E.Measuring Colorfulness in Na-tural Images.Proc of the SPIE,2003,5007:87-95.