Person Re-identification Based on Fusion Relationship Learning Network
WU Ziqiang1, CHANG Hong1,2, MA Bingpeng1
1. School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049 2. Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190
Abstract There are two problems in person re-identification methods based on graph convolutional network(GCN). While graphs are built for feature maps, the semantic information represented by graph node is not salient. The process of selecting feature blocks to build graph just relies on the relative distance among feature blocks, and the content similarity is ignored. To settle these two problems, an algorithm of person re-identification based on fusion relationship learning network(FRLN) is proposed in this paper. By employing attention mechanism, the maximum attention model makes the most important feature block more salient and assigns semantic information to it. The affinities of feature blocks are evaluated by the fusion similarity metric in the aspect of distance and content, and thus the metric is more comprehensive. By the proposed algorithm, the neighbor feature blocks are selected comprehensively and better input graph structures are provided for GCN. Hence, more robust structure relationship features are extracted by GCN. Experiments on iLIDS-VID and MARS datasets verify the effectiveness of FRLN.
Fund:National Natural Science Foundation of China(No.61876171,61976203), Open Project Fund from Shenzhen Institute of Artificial Intelligence and Robotics for Society (No.AC01 202005015)
Corresponding Authors:
MA Bingpeng, Ph.D., associate professor. His research interests include principles and applications of pattern recognition, computer vision, artificial intelligence and machine learning.
About author:: WU Ziqiang, master student. His research interests include person re-identification and natural language-based person search. CHANG Hong, Ph.D., professor. Her research interests include algorithms and models in machine learning and pattern recognition, and their applications in image processing, computer vision and data mining.
[1] GRAY D, BRENNAN S, TAO H. Evaluating Appearance Models for Recognition, Reacquisition, and Tracking[C/OL]. [2021-04-26]. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.331.7285&rep=rep1&type=pdf. [2] LIAO S C, HU Y, ZHU X Y, et al. Person Re-identification by Local Maximal Occurrence Representation and Metric Learning[C/OL]. [2021-04-26]. https://arxiv.org/pdf/1406.4216v2.pdf. [3] MA B P, SU Y, JURIE F. Local Descriptors Encoded by Fisher Vectors for Person Re-identification // Proc of the European Confe-rence on Computer Vision. Berlin, Germany: Springer, 2012: 413-422. [4] 王 鹏,宋晓宁,吴小俊,等.用于行人重识别的多类型特征网络.模式识别与人工智能, 2020, 33(10): 879-888. (WANG P, SONG X N, WU X J, et al. Multi-type Features Network for Person Re-identification. Pattern Recognition and Artificial Intelligence, 2020, 33(10): 879-888.) [5] 蒋桧慧,张 荣,李小宝,等.融合直接度量和间接度量的行人再识别.模式识别与人工智能, 2018, 31(2): 167-174. (JIANG H H, ZHANG R, LI X B, et al. Pedestrian Re-identification Fusing Direct Metric and Indirect Metric. Pattern Recognition and Artificial Intelligence, 2018, 31(2): 167-174.) [6] KÖSTINGER M, HIRZER M, WOHLHART P, et al. Large Scale Metric Learning from Equivalence Constraints // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2012: 2288-2295. [7] SUN Y F, ZHENG L, YANG Y, et al. Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline) // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 501-518. [8] ZHAO L M, LI X, ZHUANG Y T, et al. Deeply-Learned Part-Aligned Representations for Person Re-identification // Proc of the IEEE International Conference on Computer Vision. Berlin, Ger-many: Springer, 2017: 3239-3248. [9] ZHENG Z D, YANG X D, YU Z D, et al. Joint Discriminative and Generative Learning for Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019. 2133-2142. [10] CHEN W H, CHEN X T, ZHANG J G, et al. Beyond Triplet Loss: A Deep Quadruplet Network for Person Re-identification // Proc of the IEEE Conference on Computer Vision and Pattern Re-cognition. Washington, USA: IEEE, 2017: 1320-1329. [11] HERMANS A, BEYER L, LEIBE B. In Defense of the Triplet Loss for Person Re-identification[C/OL]. [2021-04-26]. https://arxiv.org/pdf/1703.07737v1.pdf. [12] YU R, DOU Z Y, BAI S, et al. Hard-Aware Point-to-Set Deep Metric for Person Re-identification // Proc of the European Confe-rence on Computer Vision. Berlin, Germany: Springer, 2018: 196-212. [13] LÜ J M, CHEN W H, LI Q, et al. Unsupervised Cross-Dataset Person Re-identification by Transfer Learning of Spatial-Temporal Patterns // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 7948-7956. [14] MENG J K, WU S, ZHENG W S. Weakly Supervised Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 760-769. [15] YU H X, ZHENG W S, WU A C, et al. Unsupervised Person Re-identification by Soft Multilabel Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 2147-2157. [16] YU H X, ZHENG W S. Weakly Supervised Discriminative Feature Learning with State Information for Person Identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 5527-5537. [17] BAI S, BAI X, TIAN Q. Scalable Person Re-identification on Supervised Smoothed Manifold // Proc of the IEEE International Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 2530-2539. [18] WANG G A, YANG S, LIU H Y, et al. High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 6448-6457. [19] KIPF T N, WELLING M. Semi-supervised Classification with Graph Convolutional Networks[C/OL]. [2021-04-26]. https://arxiv.org/pdf/1609.02907.pdf. [20] BRUNA J, ZAREMBA W, SZLAM A, et al. Spectral Networks and Locally Connected Networks on Graphs[C/OL]. [2021-04-26]. https://arxiv.org/pdf/1312.6203.pdf. [21] DEFFERRARD M, BRESSON X, VANDERGHEYNST P. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering // Proc of the 30th International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2016: 3844-3852. [22] BAO L Q, MA B P, CHANG H, et al. Preserving Structural Relationships for Person Re-identification // Proc of the IEEE International Conference on Multimedia and Expo Workshops. Washington, USA: IEEE, 2019: 120-125. [23] HOU R B, MA B P, CHANG H, et al. Interaction-and-Aggregation Network for Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 9317-9326. [24] WANG X L, GIRSHICK R, GUPTA A, et al. Non-local Neural Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 7794-7803. [25] YAN Y C, ZHANG Q, NI B B, et al. Learning Context Graph for Person Search // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 2158-2167. [26] LIU X H, ZHAO H Y, TIAN M Q, et al. HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 350-359. [27] LU J S, XIONG C M, PARIKH D, et al. Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning // Proc of the IEEE Conference on Computer Vision and Pattern Re-cognition. Washington, USA: IEEE, 2017: 3242-3250. [28] SUN Y F, XU Q, LI Y L, et al. Perceive Where to Focus: Lear-ning Visibility-Aware Part-Level Features for Partial Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 393-402. [29] XU K, BA J, KIROS R, et al. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention[C/OL]. [2021-04-26]. https://arxiv.org/pdf/1502.03044.pdf. [30] ZHU Z, HUANG T T, SHI B G, et al. Progressive Pose Attention Transfer for Person Image Generation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 2347-2356. [31] XU S J, CHENG Y, GU K, et al. Jointly Attentive Spatial-Temporal Pooling Networks for Video-Based Person Re-identification // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 4733-4742. [32] LI S, BAK S, CARR P, et al. Diversity Regularized Spatiotemporal Attention for Video-Based Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 369-378. [33] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional Block Attention Module // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 3-19. [34] WANG T Q, GONG S G, ZHU X T, et al. Person Re-identification by Video Ranking // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 688-703. [35] ZHENG L, BIE Z, SUN Y F, et al. MARS: A Video Benchmark for Large-Scale Person Re-identification // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 868-884. [36] BOLLE R M, CONNELL J H, PANKANTI S, et al. The Relation between the ROC Curve and the CMC // Proc of the 4th IEEE Workshop on Automatic Identification Advanced Technologies. Washington, USA: IEEE, 2005: 15-20. [37] ZHENG L, SHEN L Y, TIAN L, et al. Scalable Person Re-identification: A Benchmark // Proc of the IEEE International Confe-rence on Computer Vision. Washington, USA: IEEE, 2015: 1116-1124. [38] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778. [39] DENG J, DONG W, SOCHER R, et al. ImageNet: A Large-Scale Hierarchical Image Database // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2009: 248-255. [40] YOU J J, WU A C, LI X, et al. Top-Push Video-Based Person Re-identification // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1345-1353. [41] LIU Y, YAN J J, OUYANG W L. Quality Aware Network for Set to Set Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 4694-4703. [42] ZHOU Z, HUANG Y, WANG W, et al. See the Forest for the Trees: Joint Spatial and Temporal Recurrent Neural Networks for Video-Based Person Re-identification // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 6776-6785. [43] SONG G L, LENG B, LIU Y, et al. Region-Based Quality Estimation Network for Large-Scale Person Re-identification[C/OL]. [2021-04-26]. https://arxiv.org/pdf/1711.08766.pdf. [44] GAO C X, CHEN Y, YU J G, et al. Pose-Guided Spatiotemporal Alignment for Video-Based Person Re-identification. Information Sciences, 2020, 527: 176-190. [45] BAO L Q, MA B P, CHANG H, et al. Masked Graph Attention Network for Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2019: 1496-1505. [46] WU Y M, BOURAHLA O E F, LI X, et al. Adaptive Graph Re-presentation Learning for Video Person Re-identification. IEEE Transactions on Image Processing, 2020, 29: 8821-8830.
2021, Vol. 34 
