Multi-task Learning Based Face Analysis for Machine Bionic Eyes
FAN Di1, Hyunwoo Kim1, CHEN Xiaopeng1, LIU Yunhui2, HUANG Qiang1
1.School of Mechatronical Engineering, Beijing Institute of Technology, Beijing 100081 2.Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong
Abstract The performance of human-machine interaction is crucial for intelligence robot, and face analysis makes human-machine interaction more friendly. In this paper, a multi-task learning convolutional neural network is proposed. The tasks of smile recognition and gender classification are solved simultaneously. Inherent correlated tasks are learned, and the performance of individual task is improved. On CelebA test dataset, the proposed network achieves high accuracy on a smile recognition task and a gender classification task. The proposed model is tested on the designed machine bionic vision eyes, achieving satisfactory result on smile recognition and gender classification. The research on face analysis in this paper improves the human-machine interaction ability with the machine bionic eyes.
Fund:Supported by National Natural Science Foundation of China(No. 91748202)
Corresponding Authors:
Hyunwoo Kim, Ph.D., associate professor. His research interests include deep learning and computer vision.
About author:: FAN Di, Ph.D.candidate. His research interests include deep learning and computer vision.CHEN Xiaopeng, Ph.D., associate professor. His research interests include robotic vision and robotic control.LIU Yunhui, Ph.D., professor. His research interests include robotics,electromechanical system and computer vision.HUANG Qiang, Ph.D., professor. His research interests include biomimetic techno-logy and robotics.)
[1] TAIGMAN Y, YANG M, RANZATO M A, et al. Deepface: Clo-sing the Gap to Human-Level Performance in Face Verification // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 1701-1708. [2] SUN Y, WANG X G, TANG X O. Deep Convolutional Network Cascade for Facial Point Detection // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 3476-3483. [3] CAI H B, LIU B L, ZHANG J H, et al. Visual Focus of Attention Estimation Using Eye Center Localization. IEEE Systems Journal, 2017, 11(3): 1320-1325. [4] LIU M Y, LI S X, SHAN S G, et al. AU-Aware Deep Networks for Facial Expression Recognition // Proc of the 10th IEEE Internatio-nal Conference and Workshops on Automatic Face and Gesture Reco-gnition. Washington, USA: IEEE, 2013. DOI: 10.1109/FG.2013.6553734. [5] BALUJA S, ROWLEY H A. Boosting Sex Identification Perfor-mance. International Journal of Computer Vision, 2007, 71(1): 111-119. [6] GLAUNER P O. Deep Learning For Smile Recognition // Proc of the 12th Conference on Uncertainty Modelling in Knowledge Engineering and Decision Making. Berlin, Germany: Springer, 2016: 319-324. [7] ANTIPOV G, BERRANI S A, DUGELARY J L. Minimalistic CNN-Based Ensemble Model for Gender Prediction from Face Images. Pattern Recognition Letters, 2016, 70: 59-65. [8] CHEN D, REN S Q, WEI Y C, et al. Joint Cascade Face Detection and Alignment // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 109-122. [9] CARUANA R. Multitask Learning. Machine Learning. 1997, 28(1): 41-75. [10] EIGEN D, FERGUS R. Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture[C/OL]. [2018-08-28]. https://arxiv.org/pdf/1411.4734.pdf. [11] KOKKINOS I. UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory[C/OL]. [2018-08-28]. https://arxiv.org/pdf/1609.02132.pdf. [12] ZHU X X, RAMANAN D. Face Detection, Pose Estimation, and Landmark Localization in the Wild // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2012: 2879-2886. [13] ZHANG K H, HUANG Y Z, WU H, et al. Facial Smile Detection Based on Deep Learning Features // Proc of the 3rd IAPR Asian Conference on Pattern Recognition. Washington, USA: IEEE, 2015: 534-538. [14] MAVADATI S M, MAHOOR M H, BARTLETT K, et al. DISFA: A Spontaneous Facial Action Intensity Database. IEEE Transactions on Affective Computing, 2013, 4(2): 151-160. [15] CHEN J K, OU Q H, CHI Z R, et al. Smile Detection in the Wild with Deep Convolutional Neural Networks. Machine Vision and Applications, 2017, 28(1/2): 173-183. [16] NIAN F D, LI L Y, LI T, et al. Robust Gender Classification on Unconstrained Face Images // Proc of the 7th International Confe-rence on Internet Multimedia Computing and Service. New York, USA: ACM, 2015. DOI: 10.1145/2808492.2808570. [17] HUANG G B, MATTAR M, BERG T, et al. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments[C/OL]. [2018-08-28]. http://w.tamaraberg.com/papers/Huang_eccv2008-lfw.pdf. [18] VAN DE WOLFSHAAR J, KARAABA M F, WIERING M A. Deep Convolutional Neural Networks and Support Vector Machines for Gender Recognition // Proc of the IEEE Symposium Series on Computing Intelligence. Washington, USA: IEEE, 2015: 188-195. [19] PHILLIPS P J, WECHSLER H, HUANG J, et al. The FERET Database and Evaluation Procedure for Face-Recognition Algorithms. Image and Vision Computing, 1998, 16(5): 295-306. [20] PHILLIPS P J, MOON H, RIZVI S A, et al. The FERET Evaluation Methodology for Face-Recognition Algorithms. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 2000, 22(10): 1090-1104. [21] EIDINGER E, ENBAR R, HASSNER T. Age and Gender Estimation of Unfiltered Faces. IEEE Transactions on Information Forensics and Security, 2014, 9(12): 2170-2179. [22] MANSANET J, ALBIOL A, PAREDES R. Local Deep Neural Networks for Gender Recognition. Pattern Recognition Letters, 2016, 70: 80-86. [23] GALLAGHER A C, CHEN T. Understanding Images of Groups of People // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2009: 256-263. [24] KUINDERSMA S, DEITS R, FALLON M, et al. Optimization-Based Locomotion Planning, Estimation, and Control Design for the Atlas Humanoid Robot. Autonomous Robots, 2016, 40(3): 429-455. [25] KANEKO K, HARADA K, KANEHIRO F, et al. Humanoid Robot HRP-3 // Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington, USA: IEEE, 2008: 2471-2478. [26] SAKAGAMI Y, WATANABE R, AOYAMA C, et al. The Intelligent ASIMO: System Overview and Integration // Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington, USA: IEEE, 2002, III: 2478-2483. [27] WIKIPEDIA. Head and Neck Anatomy[J/OL]. [2018-08-28]. http://en.wikipedia.org/wiki/Head_and_neck_anatomy. [28] ZATSIORSKY V M. Kinematics of Human Motion. Champaign, USA: Human Kinetics, 1998. [29] WIKIPEDIA. Human Eye[J/OL]. [2018-08-28]. http://en.wikipedia.org/wiki/Human_eye. [30] ZEILER M D, FERGUS R. Visualizing and Understanding Convolutional Networks // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 818-833. [31] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778. [32] SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition[C/OL]. [2018-08-28]. https://arxiv.org/pdf/1409.1556.pdf. [33] LIU Z W, LUO P, WANG X G, et al. Deep Learning Face Attri-butes in the Wild // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 3730-3738. [34] ZHANG N, PALURI M, RANZATO M A, et al. Panda: Pose Aligned Networks for Deep Attribute Modeling // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Wa-shington, USA: IEEE, 2014: 1637-1644. [35] EHRLICH M, SHIELDS T J, ALMAEV T, et al. Facial Attributes Classification Using Multi-task Representation Learning // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 47-55. [36] RANJAN R, PATEL V M, CHELLAPPA R. Hyperface: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition[J/OL]. [2018-08-28]. https://arxiv.org/pdf/1603.01249.pdf.
2019, Vol. 32 
