Object Tracking with Multi-spatial Resolutions and Adaptive Feature Fusion Based on Correlation Filters
TANG Zhangyong1, WU Xiaojun2, ZHU Xuefeng1
1. School of Internet of Things Engineering, Jiangnan University, Wuxi 214122; 2. Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence, Jiangnan University, Wuxi 214122
Abstract Correlation filter(CF) based trackers cannot take advantage of the complementary characteristic of deep features and shallow features. To mitigate this problem, an object tracking algorithm with multi-spatial resolutions and adaptive feature fusion based on correlation filter is proposed. Firstly, ResNet-50 is employed to extract deep features and enhance the discrimination and robustness of feature representation during tracking. Additionally, according to the characteristic of different features with different spatial resolutions, image patches in different scales are segmented from video frame as the search area to balance the boundary effect and the number of samples. Finally, an adaptive feature fusion strategy is introduced to fuse the response maps corresponding to two kinds of features with adaptive weights to utilize the complementary characteristic. The experiments on multiple standard datasets verify the effectiveness and robustness of the proposed algorithm.
Fund:Supported by National Natural Science Foundation of China(No.61672265,U1836218), The 111 Project of Ministry of Education of China(No.B12018)
Corresponding Authors:
WU Xiaojun,Ph.D.,professor. His research interests include artificial intelligence, pattern recognition and computer vision.
About author:: TANG Zhangyong, master student. His research interests include pattern recognition and artificial intelligence.ZHU Xuefeng, Ph.D. candidate. His research interests include pattern recognition and artificial intelligence.
[1] 周俊静,段建民,杨光祖.基于改进的确定性目标关联的车辆跟踪方法.模式识别与人工智能, 2014, 27(1): 89-95. (ZHOU J J, DUAN J M, YANG G Z. Vehicle Tracking Based on Improved Deterministic Data Association. Pattern Recognition and Artificial Intelligence, 2014, 27(1): 89-95.) [2] 吴小俊,徐天阳,须文波.基于相关滤波的视频目标跟踪算法综述.指挥信息系统与技术, 2017, 8(3): 1-5. (WU X J, XU T Y, XU W B. Review of Target Tracking Algorithms in Video Based on Correlation Filter. Command Information System and Technology, 2017, 8(3): 1-5.) [3] DAI K N, WANG D, LU H C, et al. Visual Tracking via Adaptive Spatially-Regularized Correlation Filters // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 4670-4679. [4] 张 微,康宝生.相关滤波目标跟踪进展综述.中国图象图形学报, 2017, 22(8): 1017-1033. (ZHANG W, KANG B S. Recent Advances in Correlation Filter-Based Object Tracking: A Review. Journal of Image and Graphics, 2017, 22(8): 1017-1033.) [5] 卢湖川,李佩霞,王 栋.目标跟踪算法综述.模式识别与人工智能, 2018, 31(1): 61-76. (LU H Z, LI P X, WANG D. Visual Object Tracking: A Survey. Pattern Recognition and Artificial Intelligence, 2018, 31(1): 61-76.) [6] 冯 棐,吴小俊,徐天阳.基于子空间和直方图的多记忆自适应相关滤波目标跟踪算法.模式识别与人工智能, 2018, 31(7): 612-624. (FENG F, WU X J, XU T Y. Object Tracking with Multiple Me-mory Learning and Adaptive Correlation Filter Based on Subspace and Histogram. Pattern Recognition and Artificial Intelligence, 2018, 31(7): 612-624.) [7] BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual Object Tracking Using Adaptive Correlation Filters // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Re-cognition. Washington, USA: IEEE, 2010: 2544-2550. [8] HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the Circulant Structure of Tracking-by-Detection with Kernels // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2012: 702-715. [9] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-Speed Tracking with Kernelized Correlation Filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583-596. [10] DALAL N, TRIGGS B. Histograms of Oriented Gradients for Human Detection // Proc of the International Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2005, 1: 886-893. [11] DANELLJAN M, KHAN F S, FELSBERG M, et al. Adaptive Co-lor Attributes for Real-Time Visual Tracking // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 1090-1097. [12] LI Y, ZHU J K. A Scale Adaptive Kernel Correlation Filter Trac-ker with Feature Integration // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 254-265. [13] DANELLJAN M, HÄGER G, KHAN F S, et al. Accurate Scale Estimation for Robust Visual Tracking // Proc of the British Machine Vision Conference. Guildford, UK: BMVA Press, 2014. DOI: 10.5244/C.28.65. [14] DANELLJAN M, HÄGER G, KHAN F S, et al. Learning Spatially Regularized Correlation Filters for Visual Tracking // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 4310-4318. [15] GALOOGAHI H K, SIM T, LUCEY S. Correlation Filters with Limited Boundaries // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 4630-4638. [16] LUKEZˇICˇ A, VOJIRˇ T, ZAJC L C, et al. Discriminative Correlation Filter with Channel and Spatial Reliability // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 4847-4856. [17] LI F, TIAN C, ZUO W M, et al. Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 4904-4913. [18] XU T Y, FENG Z H, WU X J, et al. Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Tracking. IEEE Transactions on Image Processing, 2019, 28(11): 5596-5609. [19] XU T Y, FENG Z H, WU X J, et al. Joint Group Feature Selection and Discriminative Filter Learning for Robust Visual Object Tracking // Proc of the IEEE International Conference on Compu-ter Vision. Washington, USA: IEEE, 2019: 7950-7960. [20] SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition[C/OL]. [2019-10-25]. https://arxiv.org/pdf/1409.1556.pdf. [21] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778. [22] BHAT G, JOHNANDER J, DANELLJAN M, et al. Unveiling the Power of Deep Tracking // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 493-509. [23] DANELLJAN M, HÄGER G, KHAN F S, et al. Discriminative Scale Space Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(8): 1561-1575. [24] BOYD S, PARIKH N, CHU E, et al. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning, 2011, 3(1): 1-122. [25] WU Y, LIM J, YANG M H. Object Tracking Benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834-1848. [26] LIANG P P, BLASCH E, LING H B. Encoding Color Information for Visual Tracking: Algorithms and Benchmark. IEEE Transactions on Image Processing, 2015, 24(12): 5630-5644. [27] MUELLER M, SMITH N, GHANEM B. A Benchmark and Simulator for UAV Tracking // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 445-461. [28] KRISTAN M, LEONARDIS A, MATAS J, et al. The Sixth Visual Object Tracking Vot2018 Challenge Results // Proc of the Euro-pean Conference on Computer Vision. Berlin, Germany: Springer, 2018: 3-53. [29] DANELLJAN M, BHAT G, KHAN F S, et al. ECO: Efficient Convolution Operators for Tracking // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 6931-6939. [30] GALOOGAHI H K, FAGG A, LUCEY S. Learning Background-Aware Correlation Filters for Visual Tracking // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 1144-1152. [31] BERTINETTO L, VALMADRE J, GOLODETZ S, et al. Staple: Complementary Learners for Real-Time Tracking // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1404-1409. [32] MUELLER M, SMITH N, GHANEM B. Context-Aware Correla- tion Filter Tracking // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 1378-1395. [33] DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 472-488. [34] SUN C, WANG D, LU H C, et al. Correlation Tracking via Joint Discrimination and Reliability Learning[C/OL]. [2019-10-25]. https://arxiv.org/pdf/1804.08965v1.pdf. [35] SUN C, WANG D, LU H C, et al. Learning Spatial-Aware Regressions for Visual Tracking[C/OL]. [2019-10-25]. https://arxiv.org/pdf/1706.07457.pdf.
2020, Vol. 33 
