Human Action Recognition Using RGB-D Image Features
TANG Chao1, WANG Wenjian2, ZHANG Chen1, PENG Hua3, LI Wei4
1.School of Artificial Intelligence and Big Data, Hefei University, Hefei 230601; 2.School of Computer and Information Technology, Shanxi University, Taiyuan 030006; 3.Department of Computer Science and Engineering, Shaoxing University, Shaoxing 312000; 4.School of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361024
Abstract Since the existing multi-modal feature fusion methods cannot measure the contribution of different features effectively, a human action recognition method based on RGB-depth image features is proposed. Firstly, the histogram of oriented gradient feature based on RGB modal information, the space-time interest points feature based on depth modal information, and the joints relative position feature based on joints modal information are acquired to express human actions, respectively. Then, nearest neighbor classifiers with different distance measurement formulas are utilized to classify prediction samples expressed by the three modal features. The experimental results on public datasets show that the proposed method is simple, fast and efficient.
Fund:Supported by National Natural Science Foundation of China(No.61673249,61806068,61662025), Excellent Talents Training Project of Universities of Anhui Province(No.gxfx2017099), Scholarship for Studying Abroad Program of Fujian, Science and Technology Planning Guidance Project of Xiamen(No.3502Z20179038), Key Teaching and Research Project of Hefei University(No.2018 hfjyxm09)
Corresponding Authors:
TANG Chao, Ph.D., associate professor. His research interests include machine learning and compu-ter vision.
About author:: WANG Wenjian, Ph.D., professor. Her research interests include machine learning and computer intelligence;ZHANG Chen, Ph.D., lecturer. Her research interests include machine learning and computer intelligence;PENG Hua, Ph.D., lecturer. His research interests include brain-like intelligent systems, human-robot interaction and machine learning;LI Wei, Ph.D., associate professor. His research interests include artificial Intelligence, computer graphics and human compu-ter interaction.
[1] CHEN C, JAFARI R, KEHTARNAVAZ N. A Survey of Depth and Inertial Sensor Fusion for Human Action Recognition. Multimedia Tools and Applications, 2017, 76(3): 4405-4425. [2] CAI Z Y, HAN J G, LIU L, et al. RGB-D Datasets Using Microsoft Kinect or Similar Sensors: A Survey. Multimedia Tools and Applications, 2017, 76(3): 4313-4355. [3] ZHANG J, LI W Q, OGUNBONA P O, et al. RGB-D-Based Action Recognition Datasets: A Survey. Pattern Recognition, 2016, 60: 86-105. [4] ZHANG B C, YANG Y, CHEN C, et al. Action Recognition Using 3D Histograms of Texture and a Multi-class Boosting Classifier. IEEE Transactions on Image Processing, 2017, 26(10): 4648-4660. [5] CHEN C, LIU M Y, LIU H, et al. Multi-temporal Depth Motion Maps-Based Local Binary Patterns for 3-D Human Action Recognition. IEEE Access, 2017, 5: 22590-22604. [6] CHEN C, LIU K, KEHTARNAVAZ N. Real-Time Human Action Recognition Based on Depth Motion Maps. Journal of Real-Time Image Processing, 2016, 12(1): 155-163. [7] LI W Q, ZHANG Z Y, LIU Z C. Action Recognition Based on a Bag of 3D Points // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2010: 9-14. [8] YANG X D, ZHANG C Y, TIAN Y L. Recognizing Actions Using Depth Motion Maps-Based Histograms of Oriented Gradients // Proc of the 20th ACM International Conference on Multimedia. New York, USA: ACM, 2012 : 1057-1060. [9] BULBUL M F, JIANG Y S, MA J W. Human Action Recognition Based on DMMs, HOGs and Contourlet Transform // Proc of the IEEE International Conference on Multimedia Big Data. Washington, USA: IEEE, 2015: 389-394. [10] YANG X D, TIAN Y L. Super Normal Vector for Activity Recognition Using Depth Sequences // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 804-811. [11] SLAMA R, WANNOUS H, DAOUDI M. Grassmannian Representation of Motion Depth for 3D Human Gesture and Action Recognition // Proc of the 22nd International Conference on Pattern Recognition. Washington, USA: IEEE, 2014: 3499-3504. [12] OREIFEJ O, LIU Z C. HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013 : 716-723. [13] JIA C C, FU Y. Low-Rank Tensor Subspace Learning for RGB-D Action Recognition. IEEE Transactions on Image Processing, 2016, 25(10): 4641-4652. [14] LEIGHTLEY D, MCPHEE J S, YAP M H. Automated Analysis and Quantification of Human Mobility Using a Depth Sensor. IEEE Journal of Biomedical and Health Informatics, 2017, 21(4): 939-948. [15] KEROLA T, INOUE N, SHINODA K. Cross-View Human Action Recognition from Depth Maps Using Spectral Graph Sequences. Computer Vision and Image Understanding, 2017, 154: 108-126. [16] DALAL N, TRIGGS B. Histograms of Oriented Gradients for Human Detection // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2005, I: 886-893. [17] LAPTEV I, LINDEBERG T. Space-Time Interest Points // Proc of the 9th IEEE International Conference on Computer Vision. Wa-shington, USA: IEEE, 2003, I: 432-439. [18] HARRIS C, STEPHENS M, HARRIS C J, et al. A Combined Corner and Edge Detector // Proc of the 4th Alvey Vision Confe-rence. Berlin, Germany: Springer, 1988: 147-151. [19] JOHANSSON G. Visual Perception of Biological Motion and a Model for Its Analysis. Perception & Psychophysics, 1973, 14(2): 201-211.
2019, Vol. 32 
