Abstract To recognize gestures of multi-touch system, a framework including gesture description and recognition is proposed. Multi-touch gesture can be decomposed into atomic gestures and composite gestures. For gesture description, the back propagation neural networks (BPNN) is used to model the atomic gesture. Users motions are mapped into composite gestures combined with atomic gestures logic, temporal and spatial relations. Petri nets (PN) introduced with logical, temporal, spatial descriptors are used to model the composite gesture. For recognition, BPNN is used as a classifier to recognize atomic gestures and the recognition results trigger the transition of Petri nets for composite gestures to realize the recognition. Experimental results show that the proposed method is robust to different users customs and can recognize multi-touch gestures effectively.
WANG De-Xin,SHI Chong-Lin,ZHANG Mao-Jun. Multi-Touch Gesture Recognition Based on Petri Net and Back Propagation Neural Networks[J]. , 2010, 23(3): 408-413.
[1] Wang Dexin, Zhang Maojun, Xiong Zhihui. Survey on Multi-Touch Research. Application Research of Computers, 2009, 26(7): 2404-2406 (in Chinese) (王德鑫,张茂军,熊志辉.多重触控技术研究综述.计算机应用研究, 2009, 26(7): 2404-2406) [2] Popa D, Simion G, Gui V, et al. Real Time Trajectory Based Hand Gesture Recognition. WSEAS Trans on Information Science and Applications, 2008, 5(4): 532-546 [3] Alon J, Athitsos V, Yuan Quan, et al. Simultaneous Localization and Recognition of Dynamic Hand Gestures // Proc of the IEEE Workshop on Motion and Video Computing. Breckenridge, USA, 2005, Ⅱ: 254-260 [4] Yang M H, Ahuja N, Tabb M. Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition. IEEE Trans on Pattern Analyze and Machine Intelligence, 2002, 24(8): 1061-1074 [5] Corradini A G, Gross H M. Implementation and Comparison of Three Architectures for Gesture Recognition // Proc of the IEEE International Conference on Acoustics, Speech, and Signal Processing. Istanbul, Turky, 2000: 2361-2364 [6] Lichtenauer J F, Hendriks E A, Reinders M J. Sign Language Recognition by Combining Statistical DTW and Independent Classification. IEEE Trans on Pattern Analysis and Machine Intelligence, 2008, 30(11): 2040-2046 [7] Ren Haibing, Zhu Yuanxin, Xu Guangyou, et al. Spatio-Temporal Appearance Modeling and Recognition of Continuous Dynamic Hand Gestures. Chinese Journal of Computers, 2000, 23(8): 824-828 (in Chinese) (任海兵,祝远新,徐光祐,等.连续动态手势的时空表观建模及识别.计算机学报, 2000, 23(8): 824-828) [8] Wu Ying, Huang T S. View-Independent Recognition of Hand Posture // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Hilton Head Island, USA, 2000, Ⅱ: 88-94 [9] Xu Deyou. A Neural Network Approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG // Proc of the 18th International Conference on Pattern Recognition. Hongkong, China, 2006, Ⅲ: 519-522 [10] Yang M H, Ahuja N. Recognizing Hand Gesture Using Motion Trajectories // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Fort Collins, USA, 1999, Ⅰ: 466-472 [11] Lee H K, Kim J H. An HMM-Based Threshold Model Approach for Gesture Recognition. IEEE Trans on Pattern Analysis and Machine Intelligence, 1999, 21(10): 961-973 [12] Wah N C, Ranganath S. Real-Time Gesture Recognition System and Application. Image and Vision Computing, 2002, 20(13): 993-1007 [13] Yang R, Sarkar S. Gesture Recognition Using Hidden Markov Models from Fragmented Observations // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA, 2006, Ⅰ: 766-773 [14] Wang Xiying, Dai Guozhong, Zhang Xiwen, et al. Recognition of Complex Dynamic Gesture Based on HMM-FNN Model. Journal of Software, 2008, 19(9): 2302-2312 (in Chinese) (王西颖,戴国忠,张习文,等.基于HMM-FNN模型的复杂动态手势识别, 2008, 19(9): 2302-2312) [15] Liu Jianghua, Chen Jiaping, Cheng Junshi. Dynamic Hand Gesture Recognition Based on Optical Flow and Coupled Hidden Markov Model. Journal of Shanghai Jiaotong University, 2003, 37(5): 720-723 (in Chinese) (刘江华,陈佳品,程君实.基于光流及耦合隐马尔可夫模型的动态手势识别.上海交通大学学报, 2003, 37(5): 720-723) [16] Yuan Chongyi. Petri Nets Theory and Applications.Beijing, China: Electronics Industry Press, 2005 (in Chinese) (袁崇义.Petri网原理与应用.北京:电子工业出版社, 2005) [17] Allen J F, Ferguson G. Actions and Events in Interval Temporal Logic. Journal of Logic and Computation, 1994, 4(5): 531-579 [18] Wang Dexin, Zhang Maojun, Xiong Zhihui, et al. A Multi-Touch System Using Four Cameras. Journal of Computer-Aided Design Computer Graphics, 2009, 21(10): 1507-1512 (in Chinese) (王德鑫,张茂军,熊志辉,等.四路摄像头协同的多重触控系统.计算机辅助设计与图形学学报, 2009, 21(10): 1507-1512)
2010, Vol. 23 
