Visual Tracking Algorithm Based on Rotation Adaptation, Multi-feature Fusion and Multi-template Learning
DU Chenjie1, YANG Yuxiang1, WU Han1, HE Zhiwei1, GAO Mingyu2
1. School of Electronics and Information Engineering, Hangzhou Dianzi University, Hangzhou 310018 2. Zhejiang Provincial Key Laboratory of Equipment Electronics, Hangzhou Dianzi University, Hangzhou 310018
Abstract Visual target tracking remains a hard problem due to unpredictable target rotation and external interference. To address this issue, a target tracking algorithm based on rotation adaptation, multi-feature fusion and multi-template learning(RA-MFML) is proposed. Firstly, a multi-template learning model with complementary characteristics is constructed. The global filter template is used for tracking the target. When the global filter template is determined to be contaminated by the decidable filter template, it is corrected by the modified filter template. Then, the color histogram is regarded as visual supplementary information and fused with feature map of VGGNet-19 adaptively. The discriminating ability of the global filter template for object appearance is thus improved. Finally, a rotation adaptation strategy is proposed. The improved tracking confidence is utilized for the estimation of the optimal rotation angle of the tracking box to alleviate performance degradation of the global filter template caused by target rotation. The experiment on OTB-2013 and OTB-2015 datasets demonstrate that RA-MFML is superior in success rate and precision.
Fund:National Natural Science Foundation of China(No.6187307,61401129), Key Research and Development Program of Zhejiang Province(No.2018C01069)
Corresponding Authors:
HE Zhiwei, Ph.D., professor. His research interests include machine learning and object tracking.
About author:: DU Chenjie, Ph.D. candidate. His research interests include object detection and tracking. YANG Yuxiang, Ph.D., associate profe-ssor. His research interests include deep learning and object tracking. WU Han, master student. His research interests include object detection and tracking. GAO Mingyu, Ph.D., professor. His research interests include video/image proce-ssing.
DU Chenjie,YANG Yuxiang,WU Han等. Visual Tracking Algorithm Based on Rotation Adaptation, Multi-feature Fusion and Multi-template Learning[J]. , 2021, 34(9): 787-797.
[1] QI Y K, ZHANG S P, QIN L, et al. Hedging Deep Features for Visual Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(5): 1116-1130. [2] 储 珺,危 振,缪 君,等.基于遮挡检测和多块位置信息融合的分块目标跟踪算法.模式识别与人工智能, 2020, 33(1): 59-65. (CHU J, WEI Z, MIAO J, et al. Block Target Tracking Based on Occlusion Detection and Multi-block Position Information Fusion. Pattern Recognition and Artificial Intelligence, 2020, 33(1): 59-65) [3] 刘巧元,王玉茹,张金玲,等.基于相关滤波器的视频跟踪方法研究进展.自动化学报, 2019, 45(2): 265-275. (LIU Q Y, WANG Y R, ZHANG J L, et al. Research Progress of Visual Tracking Methods Based on Correlation Filter. Acta Automa-tica Sinica, 2019, 45(2): 265-275.) [4] BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual Object Tracking Using Adaptive Correlation Filters // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Re-cognition. Washington, USA: IEEE, 2010: 2544-2550. [5] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-Speed Tracking with Kernelized Correlation Filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583-596. [6] DANELLJAN M, HÄGER G, KHAN F S, et al. Learning Spatially Regularized Correlation Filters for Visual Tracking // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 4310-4318. [7] WANG J, LIU W B, XING W W, et al. Visual Object Tracking with Multi-scale Superpixels and Color-Feature Guided Kernelized Correlation Filters. Signal Processing: Image Communication, 2018, 63: 44-62. [8] YAN J R, ZHONG L C, YAO Y B, et al. Dual-Template Adaptive Correlation Filter for Real-Time Object Tracking. Multimedia Tools and Applications, 2021, 80: 2355-2376. [9] DANELLJAN M, HAGER G, KHAN F S, et al. Convolutional Features for Correlation Filter Based Visual Tracking // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 621-629. [10] MA C, HUANG J B, YANG X K, et al. Hierarchical Convolutional Features for Visual Tracking // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 3074-3082. [11] MA C, HUANG J B, YANG X K, et al. Robust Visual Tracking via Hierarchical Convolutional Features. IEEE Transactions on Pa-ttern Analysis and Machine Intelligence, 2019, 41(11): 2709-2723. [12] DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond Co-rrelation Filters: Learning Continuous Convolution Operators for Visual Tracking // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 472-488. [13] CHOI J, CHANG H J, FISCHER T, et al. Context-Aware Deep Feature Compression for High-Speed Visual Tracking // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 479-488. [14] WANG Y, WEI X, SHEN H, et al. Adaptive Model Updating for Robust Object Tracking. Signal Processing: Image Communication, 2020, 80. DOI: 10.1016/j.image.2019.115656. [15] GAO T Z, WANG N, CAI J, et al. Explicitly Exploiting Hierarchical Features in Visual Object Tracking. Neurocomputing, 2020, 397: 203-211. [16] WANG M M, LIU Y, HUANG Z Y. Large Margin Object Tracking with Circulant Feature Maps // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 4800-4808. [17] WOJKE N, BEWLEY A, PAULUS D. Simple Online and Realtime Tracking with a Deep Association Metric // Proc of the IEEE International Conference on Image Processing. Washington, USA: IEEE, 2017: 3645-3649. [18] 汤张泳,吴小俊,朱学峰.多空间分辨率自适应特征融合的相关滤波目标跟踪算法.模式识别与人工智能, 2020, 33(1): 66-74. (TANG Z Y, WU X J, ZHU X F. Object Tracking with Multi-spatial Resolutions and Adaptive Feature Fusion Based on Correlation Filters. Pattern Recognition and Artificial Intelligence, 2020, 33(1): 66-74.) [19] LI Y, LIU G Z. Learning a Scale-and-Rotation Correlation Filter for Robust Visual Tracking // Proc of the IEEE International Conference on Image Processing. Washington, USA: IEEE, 2016: 454-458. [20] 熊 丹,卢惠民,肖军浩,等.具有尺度和旋转适应性的长时间目标跟踪.自动化学报, 2019, 45(2): 289-304. (XIONG D, LU H M, XIAO J H, et al. Robust Long-Term Object Tracking with Adaptive Scale and Rotation Estimation. Acta Automatica Sinica, 2019, 45(2): 289-304.) [21] 瑚 琦,李 锐,张 薇.具有旋转特性的目标跟踪算法.光学学报, 2020, 40(17): 140-147. (HU Q, LI R, ZHANG W. Target Tracking Algorithm with Rotation Characteristics. Acta Optica Sinica, 2020, 40(17): 140-147.) [22] VEDALDI A, LENC K. MatConvNet: Convolutional Neural Networks for MATLAB // Proc of the 23rd ACM International Confe-rence on Multimedia. New York, USA: ACM, 2015: 689-692. [23] WU Y, LIM J, YANG M H. Online Object Tracking: A Benchmark // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 2411-2418. [24] WU Y, LIM J, YANG M H. Object Tracking Benchmark. IEEE Tran-sactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834-1848. [25] ZHANG Z P, PENG H W. Deeper and Wider Siamese Networks for Real-Time Visual Tracking // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 4586-4595. [26] LI B, YAN J J, WU W, et al. High Performance Visual Tracking with Siamese Region Proposal Network // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 8971-8980. [27] DONG X P, SHEN J B. Triplet Loss in Siamese Network for Object Tracking // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 472-488. [28] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-Convolutional Siamese Networks for Object Tracking // Proc of the European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2016: 850-865. [29] LUKEŽIC A, VOJÍR T, ZAJC L C, et al. Discriminative Correlation Filter with Channel and Spatial Reliability // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 4847-4856.
2021, Vol. 34 
