Surface Electromyography Classification Method Based on Temporal Two-Dimensionalization and Convolution Feature Fusion
LUO Junjin1, WANG Wanliang1, WANG Zheng1, LIU Honghai2
1. College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310014 2. Intelligent Systems and Biomedical Robotics Group, School of Computing, University of Portsmouth, UK PO1 3AH
Abstract The traditional pattern recognition methods are prone to ignore characteristics of non-linearity and timing in the classification of surface electromyography(sEMG). Aiming at this problem, a sEMG signal classification method based on temporal two-dimensionalization and convolution feature fusion is proposed. Temporal two-dimensionalization is realized by Gramian angular field conversion to preserve the time dependence and correlation of original time series of sEMG. To highlight the local information and fully retain details simultaneously, a capsule network and a convolutional neural network are introduced to extract features together. In addition, the feature fusion is performed to realize the gesture recognition under different conditions. Experimental results show that the proposed method is more robust than other classification methods and it effectively enhances the electrode offset and the overall recognition level of hand movements facing new objects.
Fund:Supported by National Natural Science Foundation of China(No.61873240)
Corresponding Authors:
WANG Wanliang, Ph.D., professor. His research interests include deep learning, artificial intelligence and big data.
About author:: LUO Junjin, master student. Her research interests include pattern recognition and deep learning.WANG Zheng, Ph.D. candidate. His research interests include intelligent computing and intelligent system.LIU Honghai, Ph.D., professor. His research interests include artificial intelligence, human-machine interaction and robotics.
LUO Junjin,WANG Wanliang,WANG Zheng等. Surface Electromyography Classification Method Based on Temporal Two-Dimensionalization and Convolution Feature Fusion[J]. , 2020, 33(7): 588-599.
[1] CORDELLA F, CIANCIO A L, SACCHETTI R, et al. Literature Review on Needs of Upper Limb Prosthesis Users. Frontiers in Neuroscience, 2016. DOI: 10.3389/fnins.2016.00209. [2] ZECCA M, MICERA S, CARROZZA M C, et al. Control of Multifunctional Prosthetic Hands by Processing the Electromyographic Signal. Critical Reviews in Biomedical Engineering, 2002, 30(4/5/6):459-485. [3] 吴冬梅,孙 欣,张志成,等.表面肌电信号的分析和特征提取.中国组织工程研究与临床康复, 2010, 14(43): 8073-8076. (WU D M, SUN X, ZHANG Z C, et al. Feature Collection and Analysis of Surface Electromyography Signals. Journal of Clinical Rehabilitative Tissue Engineering Research, 2010, 14(43): 8073-8076.) [4] PHINYOMARK A, PHUKPATTARANONT P, LIMSAKUL C. Feature Reduction and Selection for EMG Signal Classification. Expert Systems with Applications, 2012, 39(8): 7420-7431. [5] DOULAH A B M S U, FATTAH S A, ZHU W P, et al. Wavelet Domain Feature Extraction Scheme Based on Dominant Motor Unit Action Potential of EMG Signal for Neuromuscular Disease Classification. IEEE Transactions on Biomedical Circuits and Systems, 2014, 8(2): 155-164. [6] POTLURI C, KUMAR P, ANUGOLU M, et al. Frequency Domain Surface EMG Sensor Fusion for Estimating Finger Forces // Proc of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Washington, USA: IEEE, 2010:5975-5978. [7] YU Y P, FAN L C, KUANG S L,et al. The Research of sEMG Movement Pattern Classification Based on Multiple Fused Wavelet Function // Proc of the IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems. Wa-shington, USA: IEEE, 2015: 487-491. [8] MATSUBARA T, MORIMOTO J. Bilinear Modeling of EMG Signals to Extract User-Independent Features for Multiuser Myoelectric Interface. IEEE Transactions on Biomedical Engineering, 2013, 60(8):2205-2213. [9] AL-FAIZ M Z, ALI A A, MIRY A H. A k-Nearest Neighbor Based Algorithm for Human Arm Movements Recognition Using EMG Signals // Proc of the 1st International Conference on Energy, Power and Control. Berlin, Germany: Springer, 2010: 159-167. [10] PURUSHOTHAMAN G, VIKAS R. Identification of a Feature Selection Based Pattern Recognition Scheme for Finger Movement Recognition from Multichannel EMG Signals. Australasian Physical and Engineering Sciences in Medicine, 2018, 41(2): 549-559. [11] BELLINGEGNI A D, GRUPPIONI E, COLAZZO G, et al. NLR, MLP, SVM, and LDA: A Comparative Analysis on EMG Data from People with Trans-radial Amputation. Journal of Neuroengineering and Rehabilitation, 2017, 14(1). DOI: 10.1186/s12984-017-0290-6. [12] SONG G, WANG Y C, WANG M M, et al. Lower Limb Movement Intent Recognition Based on Grid Search Random Forest Algorithm // Proc of the 3rd International Conference on Robotics, Control and Automation. Berlin, Germany: Springer, 2018: 225-229. [13] 丁其川,熊安斌,赵新刚,等.基于表面肌电的运动意图识别方法研究及应用综述.自动化学报, 2016, 42(1): 13-25. (DING Q C, XIONG A B, ZHAO X G, et al. A Review on Researches and Applications of sEMG-Based Motion Intent Recognition Methods. Acta Automatica Sinica, 2016, 42(1): 13-25.) [14] KIM K T, PARK K H, LEE S W. An Adaptive Convolutional Neural Network Framework for Multi-user Myoelectric Interfaces // Proc of the 4th IAPR Asian Conference on Pattern Recognition. Washington, USA: IEEE, 2017: 788-792. [15] HE Y N, FUKUDA O, BU N, et al. Surface EMG Pattern Recognition Using Long Short-Term Memory Combined with Multilayer Perceptron // Proc of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Washington, USA: IEEE, 2018: 5636-5639. [16] WANG Z G, OATES T. Encoding Time Series as Images for Visual Inspection and Classification Using Tiled Convolutional Neural Networks // Proc of the 29th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2015: 40-46. [17] HATAMI N, GAVET Y, DEBAYLE J. Classification of Time-Series Images Using Deep Convolutional Neural Networks[C/OL]. [2010-03-05]. https://arxiv.org/pdf/1710.00886.pdf. [18] SABOUR S, FROSST N, HINTON G E. Dynamic Routing between Capsules[C/OL]. [2010-03-05]. https://arxiv.org/pdf/1710.09829.pdf. [19] HINTON G E, KRIZHEVSKY A, WANG S D. Transforming Auto-encoders // Proc of the 21st International Conference on Artificial Neural Networks. Berlin, Germany: Springer. 2011: 44-51. [20] 刘渭滨,邹智元,邢薇薇.模式分类中的特征融合方法.北京邮电大学学报, 2017, 40(4): 1-8. (LIU W B, ZOU Z Y, XING W W. Feature Fusion Methods in Pattern Classification. Journal of Beijing University of Posts and Telecommunications, 2017, 40(4): 1-8.) [21] FARINA D, JIANG N, REHBAUM H, et al. The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges. IEEE Tran-sactions on Neural Systems and Rehabilitation Engineering, 2014, 22(4): 797-809. [22] HARGROVE L, ENGLEHART K, HUDGINS B. A Training Stra-tegy to Reduce Classification Degradation due to Electrode Displacements in Pattern Recognition Based Myoelectric Control. Biomedical Signal Processing and Control, 2008, 3(2): 175-180. [23] TKACH D, HUANG H, KUIKEN T A. Study of Stability of Time-Domain Features for Electromyographic Pattern Recognition. Jour-nal of Neuroengineering and Rehabilitation, 2010. DOI: 10.1186/1743-0003-7-21. [24] FARINA D, MERLETTI R, ENOKA R M. The Extraction of Neural Strategies from the Surface EMG: An Update. Journal of Applied Physiology, 2004, 96(4): 1486-1495. [25] CHEN H F, TONG R Z, CHEN M J, et al. A Hybrid CNN-SVM Classifier for Hand Gesture Recognition with Surface EMG Signals // Proc of the International Conference on Machine Learning and Cybernetics. Washington, USA: IEEE, 2018, II: 619-624. [26] BIAN F F, LI R F, LIANG P. SVM Based Simultaneous Hand Movements Classification Using sEMG Signals // Proc of the IEEE International Conference on Mechatronics and Automation. Washington, USA: IEEE, 2017: 427-432. [27] CHEN M, CHENG L, HUANG F B, et al. Towards Robot-Assisted Post-Stroke Hand Rehabilitation: Fugl-Meyer Gesture Recognition Using sEMG // Proc of the 7th IEEE Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems. Washington, USA: IEEE, 2017: 1472-1477. [28] DUAN N, LIU L Z, YU X J, et al. Classification of Multichannel Surface-Electromyography Signals Based on Convolutional Neural Networks. Journal of Industrial Information Integration, 2019, 15:201-206. [29] ENGLEHART K, HUDGIN B, PARKER P A. A Wavelet-Based Continuous Classification Scheme for Multifunction Myoelectric Control. IEEE Transactions on Biomedical Engineering, 2001, 48(3):302-311. [30] CHU J U, MOON I, LEE Y J, et al. A Supervised Feature-Projection-Based Real-Time EMG Pattern Recognition for Multifunction Myoelectric Hand Control. IEEE/ASME Transactions on Mechatronics, 2007, 12(3): 282-290. [31] HAKONEN M, PIITULAINEN H, VISALA A. Current State of Digital Signal Processing in Myoelectric Interfaces and Related Applications. Biomedical Signal Processing and Control, 2015, 18:334-359.
2020, Vol. 33 
