Clothes-Changing Person Re-identification Method Based on Text-Image Mutual Learning
GE Bin1, LU Yang1, XIA Chenxing1, GUAN Junming2
1. School of Computer Science and Engineering, Anhui University of Science and Technology, Huainan 232001; 2. School of Information Engineering, Huangshan University, Huangshan 245041
Abstract To address the issue of low recognition accuracy in pedestrian re-identification(Re-ID) tasks involving clothing changes, a method for clothes-changing person re-identification based on text-image mutual learning(TIML) is proposed. It leverages the ability of contrastive language-image pre-training to generate pseudo-texts. In the first training phase, a pseudo-text generator is designed to enhance text diversity by swapping pixel information among samples within the same batch, thereby augmenting text variability. Additionally, a semantic alignment loss LSA is introduced to ensure the consistency in text feature representation. In the second phase of training, a global and local fusion network is devised to bolster the discriminative power of visual features by fusing local and global features, guided by the textual information obtained in the first phase. Experiments on PRCC, Celeb-ReID, Celeb-Light and VC-Clothes datasets demonstrate that the proposed model significantly improves recognition accuracy in scenarios with small dataset samples.
Fund:Supported by National Natural Science Foundation of China(No.62102003), Natural Science Foundation of Anhui Province(No.2108085QF258), Anhui Postdoctoral Science Foundation(No.2022B623), Graduate Innovation Fund of Anhui University of Science and Technology(No.2023CX2125)
Corresponding Authors:
GE Bin, Ph.D., professor. His research interests include computer vision, pattern recognition and information security.
About author:: LU Yang, Master student. His research in-terests include computer vision and deep lear-ning.XIA Chenxing, Ph.D., associate professor. His research interests include computer vision, pattern recognition and machine lear-ning.GUAN Junming, Ph.D., associate profe-ssor. His research interests include computer vision and pattern recognition.
GE Bin,LU Yang,XIA Chenxing等. Clothes-Changing Person Re-identification Method Based on Text-Image Mutual Learning[J]. Pattern Recognition and Artificial Intelligence, 2024, 37(11): 960-973.
[1] 罗浩,姜伟,范星,等.基于深度学习的行人重识别研究进展.自动化学报, 2019, 45(11): 2032-2049. (LUO H, JIANG W, FAN X, et al. A Survey on Deep Learning Based Person Re-identification. Acta Automatica Sinica, 2019, 45(11): 2032-2049.) [2] YU H X, WU A C, ZHENG W S. Unsupervised Person Re-identification by Deep Asymmetric Metric Embedding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(4): 956-973. [3] LI M X, ZHU X T, GONG S G. Unsupervised Tracklet Person Re-identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(7): 1770-1782. [4] 王鹏,宋晓宁,吴小俊,等.用于行人重识别的多类型特征网络.模式识别与人工智能, 2020, 33(10): 879-888. (WANG P, SONG X N, WU X J, et al. Multi-type Features Network for Person Re-identification. Pattern Recognition and Artificial Intelligence, 2020, 33(10): 879-888.) [5] 张磊,吴晓富,张索非,等.基于多分支协作的行人重识别网络.模式识别与人工智能, 2021, 34(9): 853-862. (ZHANG L, WU X F, ZHANG S F, et al. Multi-branch Cooperative Network for Person Re-identification. Pattern Recognition and Artificial Intelligence, 2021, 34(9): 853-862.) [6] SUN Y F, ZHENG L, YANG Y, et al. Beyond Part Models: Person Retrieval with Refined Part Pooling(and a Strong Convolutional Baseline) // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 501-518. [7] HUANG Y, WU Q, XU J S, et al. Celebrities-ReID: A Benchmark for Clothes Variation in Long-Term Person Re-identification // Proc of the International Joint Conference on Neural Networks. Washington, USA: IEEE, 2019. DOI: 10.1109/IJCNN.2019.8851957. [8] YANG Q Z, WU A C, ZHENG W S. Person Re-identification by Contour Sketch under Moderate Clothing Change. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(6): 2029-2046. [9] WAN F B, WU Y, QIAN X L, et al. When Person Re-identification Meets Changing Clothes // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2020: 3620-3628. [10] QIAN X L, WANG W X, ZHANG L, et al. Long-Term Cloth-Changing Person Re-identification // Proc of the Asian Conference on Computer Vision. Berlin, Germany: Springer, 2020: 71-88. [11] FAN C, PENG Y J, CAO C S, et al. GaitPart: Temporal Part-Based Model for Gait Recognition // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 14213-14221. [12] CHEN J X, ZHENG W S, YANG Q Z, et al. Deep Shape-Aware Person Re-identification for Overcoming Moderate Clothing Changes. IEEE Transactions on Multimedia, 2021, 24: 4285-4300. [13] HONG P X, WU T, WU A C, et al. Fine-Grained Shape-Appea-rance Mutual Learning for Cloth-Changing Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2021: 10508-10517. [14] SHU X J, LI G, WANG X, et al. Semantic-Guided Pixel Sampling for Cloth-Changing Person Re-identification. IEEE Signal Processing Letters, 2021, 28: 1365-1369. [15] GAO Z, WEI S X, GUAN W L, et al. Identity-Guided Collaborative Learning for Cloth-Changing Person Reidentification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(5): 2819-2837. [16] LI W J, HOU S H, ZHANG C J, et al. An In-Depth Exploration of Person Re-identification and Gait Recognition in Cloth-Changing Conditions // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 13824-13833. [17] ZHANG G Q, LIU J, CHEN Y H, et al. Multi-biometric Unified Network for Cloth-Changing Person Re-identification // Proc of the IEEE International Conference on Multimedia and Expo. Washington, USA: IEEE, 2022. DOI: 10.1109/ICME52920.2022.9859702. [18] LIU F Y, YE M, DU B. Dual Level Adaptive Weighting for Cloth-Changing Person Re-identification. IEEE Transactions on Image Processing, 2023, 32: 5075-5086. [19] ZHENG Z D, YANG X D, YU Z D, et al. Joint Discriminative and Generative Learning for Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 2133-2142. [20] HUANG Y, XU J S, WU Q, et al. Beyond Scalar Neuron: Adopting Vector-Neuron Capsules for Long-Term Person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(10): 3459-3471. [21] GU X Q, CHANG H, MA B P, et al. Clothes-Changing Person Re-identification with RGB Modality Only // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 1050-1059. [22] KWEON H J, CHO D. Cloth-Changing Person Re-identification with Noisy Patch Filtering. IEEE Signal Processing Letters, 2023,30: 334-338. [23] ZHANG R J, FANG Y, SONG H X, et al. Specialized Re-ran-king: A Novel Retrieval-Verification Framework for Cloth Changing Person Re-identification. Pattern Recognition, 2023, 134. DOI: 10.1016/j.patcog.2022.109070. [24] RADFORD A, KIM J W, HALLACY C, et al. Learning Transferable Visual Models from Natural Language Supervision // Proc of the 38th International Conference on Machine Learning. San Diego, USA: JMLR, 2021: 8748-8763. [25] LI S Y, SUN L, LI Q L. CLIP-ReID: Exploiting Vision-Language Model for Image Re-identification without Concrete Text Labels. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(1): 1405-1413. [26] HUANG Y, WU Q, XU J S, et al. Clothing Status Awareness for Long-Term Person Re-identification // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021, 11875-11884. [27] JIN X, HE T Y, ZHENG K C, et al. Cloth-Changing Person Re-identification from a Single Image with Gait Prediction and Regula-rization // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 14258-14267. [28] SHI W, LIU H, LIU M Y. IRANet: Identity-Relevance Aware Representation for Cloth-Changing Person Re-identification. Image and Vision Computing, 2022, 117. DOI: 10.1016/j.imavis.2021.104335. [29] MU J Y, LI Y, LI J, et al. Learning Clothes-Irrelevant Cues for Clothes-Changing Person Re-identification[C/OL].[2024-09-07]. https://bmvc2022.mpi-inf.mpg.de/0337.pdf. [30] GAO Z, WEI H W, GUAN W L, et al. Multigranular Visual-Semantic Embedding for Cloth-Changing Person Re-identification // Proc of the 30th ACM International Conference on Multimedia. New York, USA: ACM, 2022: 3703-3711. [31] BANSAL V, FORESTI G L, MARTINEL N. Cloth-Changing Person Re-identification with Self-Attention // Proc of the IEEE/CVF Winter Conference on Applications of Computer Vision. Washington, USA: IEEE, 2022: 602-610. [32] WANG Q Z, QIAN X L, FU Y W, et al. Co-attention Aligned Mutual Cross-Attention for Cloth-Changing Person Re-identification // Proc of the Asian Conference on Computer Vision. Berlin, Germany: Springer, 2022: 351-368. [33] YANG Z W, ZHONG X, LIU H, et al. Attentive Decoupling Network for Cloth-Changing Re-identification // Proc of the IEEE International Conference on Multimedia and Expo. Washington, USA: IEEE, 2022. DOI: 10.1109/ICME52920.2022.9859851. [34] LI X L, LU Y, LIU B, et al. Clothes-Invariant Feature Learning by Causal Intervention for Clothes-Changing Person Re-identification[C/OL].[2024-09-07]. https://arxiv.org/pdf/2305.06145. [35] YANG Z W, LIN M, ZHONG X, et al. Good Is Bad: Causality Inspired Cloth-Debiasing for Cloth-Changing Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 1472-1481. [36] YANG Z W, ZHONG X, ZHONG Z, et al. Win-Win by Competi-tion: Auxiliary-Free Cloth-Changing Person Re-identification. IEEE Transactions on Image Processing, 2023, 32: 2985-2999. [37] CUI Z Y, ZHOU J H, PENG Y X, et al. DCR-ReID: Deep Component Reconstruction for Cloth-Changing Person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(8): 4415-4428. [38] HAN K, GONG S G, HUANG Y, et al. Clothing-Change Feature Augmentation for Person Re-identification // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 22066-22075. [39] WU Z Y, HU Z R, DING J W. Same-Clothes Person Re-identifi-cation with Dual-Stream Network. Multimedia Systems, 2024, 30(2). DOI: 10.1007/s00530-024-01269-0. [40] TU H B, LIU C, PENG Y Y, et al. Clothing-Change Person Re-identification Based on Fusion of RGB Modality and Gait Features. Signal, Image and Video Processing, 2024, 18(3): 2367-2376. [41] WANG G S, YUAN Y F, CHEN X, et al. Learning Discriminative Features with Multiple Granularities for Person Re-identification // Proc of the 26th ACM International Conference on Multimedia. New York, USA: ACM, 2018:274-282. [42] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778. [43] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely Connected Convolutional Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 2261-2269.
2024, Vol. 37 
