Bilinear Regression Based Expression Face Neutralization Preserving Person-Specific Characters
CHEN Ying1,2, ZHANG Long-Yuan1, YI Xiao-Bin1
1.Key Laboratory of Advanced Process Control for Light Industry of Ministry of Education,Jiangnan University, Wuxi 214122 2.Key Laboratory of System Control and Information Processing of Ministry of Education,Shanghai Jiao Tong University, Shanghai 200240
Abstract In traditional learning-based expression neutralization methods, person-specific characters are removed from the visual neutral face.To solve this problem, an expression neutralization algorithm based on bilinear kernel rank reduced regression (BKRRR) is proposed. The BKRRR algorithm is designed to synthesize virtual expression and virtual neutral images from training samples simultaneously. An expression mask is established using differences of the two images. The test expression image is warped to neutral template by piece-wise affine warp. An image fusion strategy based on Poisson equation is then designed. The visual BKRRR neutral image is used as foreground, the warped image as background and the expression mask as foreground mask. And thus the visual neutralized face image with person-specific characters preserved is obtained. Experimental results show that the synthesized visual neutral image outperforms other learning-based methods in both visual and objective evaluation, and the accuracy of expression invariant face recognition is improved.
[1] Liu Y X, Schmidt K L, Cohn J F, et al. Facial Asymmetry Quantification for Expression Invariant Human Identification. Computer Vision and Image Understanding, 2003, 91(1/2): 138-159 [2] Alex A T, Asari V K, Mathew A. Gradient Feature Matching for Expression Invariant Face Recognition Using Single Reference Image // Proc of the IEEE International Conference on Systems, Man and Cybernetics. Seoul, Republic of Korea, 2012: 851-856 [3] Wang H C, Ahuja N. Facial Expression Decomposition // Proc of the 9th IEEE International Conference on Computer Vision. Nice, France, 2003, II: 958-965 [4] Bronstein A M, Bronstein M M, Kimmel R. Expression-Invariant 3D Face Recognition // Proc of the 4th International Conference on Audio- and Video-Based Biometric Person Authentication. Guildford, UK, 2003: 62-70 [5] Ming Y, Ruan Q Q. Expression-Robust 3D Face Recognition Using Bending Invariant Correlative Features. Informatica, 2011, 35(2): 231-238 [6] Blanz V, Vetter T. Face Recognition Based on Fitting a 3D Morphable Model. IEEE Trans on Pattern Analysis and Machine Intelligence, 2003, 25(9): 1063-1074 [7] Amberg B, Knothe R, Vetter T. Expression Invariant 3D Face Re- cognition with a Morphable Model // Proc of the 8th IEEE International Conference on Automatic Face and Gesture Recognition. Amsterdam, the Netherlands, 2008: 1-6 [8] Ter Haar F B, Veltkamp R C. Expression Modeling for Expression-Invariant Face Recognition. Computers and Graphics, 2010, 34(3): 231-241 [9] Asthana A, Sanderson C, Gedeon T, et al. Learning-Based Face Synthesis for Pose-Robust Recognition from Single Image // Proc of the British Machine Vision Conference. London, UK, 2009: 1-10 [10] Chai X J, Shan S G, Chen X L, et al. Locally Linear Regression for Pose-Invariant Face Recognition. IEEE Trans on Image Proce- ssing, 2007, 16(7): 1716-1725 [11] Naseem I, Togneri R, Bennamoun M. Linear Regression for Face Recognition. IEEE Trans on Pattern Analysis and Machine Intelligence, 2010, 32(11): 2106-2112 [12] Li Y L, Feng J F. Frontal Face Synthesizing According to Multiple Non-frontal Inputs and Its Application in Face Recognition. Neurocomputing, 2012, 91: 77-85 [13] Li X X, Mori G, Zhang H. Expression-Invariant Face Recognition with Expression Classification // Proc of the 3rd Canadian Conference on Computer and Robot Vision. Quebec, Canada, 2006: 77-83 [14] Cootes T F, Edwards G J, Taylor C J. Active Appearance Models. IEEE Trans on Pattern Analysis and Machine Intelligence, 2001, 23(6): 681-685
[15] Mohammadzade H, Hatzinakos D. Projection into Expression Subspaces for Face Recognition from Single Sample per Person. IEEE Trans on Affective Computing, 2013, 4(1): 69-82 [16] Han X, Yap M H, Palmer I. Face Recognition in the Presence of Expressions. Journal of Software Engineering and Applications, 2012, 5(5): 321-329 [17] Matthews I, Baker S. Active Appearance Models Revisited. International Journal of Computer Vision, 2004, 60(2): 135-164 [18] Huang D, De La Torre F. Bilinear Kernel Reduced Rank Regre- ssion for Facial Expression Synthesis // Proc of the 11th European Conference on Computer Vision. Crete, Greece, 2010, II: 364-377 [19] Bathe K J, Wilson E L. Numerical Methods in Finite Element Analysis. Englewood Cliffs, USA: Prentice-Hall, 1976 [20] Perez P, Gangnet M, Blake A. Poisson Image Editing. ACM Trans on Graphics, 2003, 22(3): 313-318 [21] Gross R, Matthews I, Cohn J, et al. Multi-PIE. Image and Vision Computing, 2010, 28(5): 807-813
2014, Vol. 27 
