基于文本图像互学习的换衣行人重识别方法

doi:10.16451/j.cnki.issn1003-6059.202411002

摘要
图/表
参考文献
相关文章 (1)

全文: PDF (4141 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要针对行人重识别在换衣场景下的小数据集样本中识别精度较低的问题,结合大模型CLIP(Contrastive Language-Image Pre-training)生成伪文本的功能,提出基于文本图像互学习的换衣行人重识别方法.在训练第一阶段,设计伪文本生成器,交换同批次中的样本像素,生成多样性文本,增强文本差异性,并通过语义对齐损失约束文本特征的一致性.在第二阶段,设计局部全局融合网络,融合局部特征和全局特征,在第一阶段文本信息的指导下,增强视觉特征的判别性.在PRCC、Celeb-ReID、Celeb-Light、VC-Clothes数据集上的实验表明,文中方法可提升在小数据集样本中的性能.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	葛斌
	卢洋
	夏晨星
	官骏鸣

关键词 ：换衣行人重识别, CLIP, 模态交互, 语义对齐, 提示工程

Abstract：To address the issue of low recognition accuracy in pedestrian re-identification(Re-ID) tasks involving clothing changes, a method for clothes-changing person re-identification based on text-image mutual learning(TIML) is proposed. It leverages the ability of contrastive language-image pre-training to generate pseudo-texts. In the first training phase, a pseudo-text generator is designed to enhance text diversity by swapping pixel information among samples within the same batch, thereby augmenting text variability. Additionally, a semantic alignment loss L_SA is introduced to ensure the consistency in text feature representation. In the second phase of training, a global and local fusion network is devised to bolster the discriminative power of visual features by fusing local and global features, guided by the textual information obtained in the first phase. Experiments on PRCC, Celeb-ReID, Celeb-Light and VC-Clothes datasets demonstrate that the proposed model significantly improves recognition accuracy in scenarios with small dataset samples.

Key words： Clothes-Changing Person Re-identification Contrastive Language-Image Pre-training(CLIP) Modal Interaction Semantic Alignment Prompt Engineering

收稿日期: 2024-10-25

ZTFLH:

TP 391.4

基金资助:国家自然科学基金项目(No.62102003)、安徽省自然科学基金项目(No.2108085QF258)、安徽省博士后科学基金项目(No.2022B623)、安徽理工大学研究生创新基金项目(No.2023CX2125)资助

通讯作者: 葛斌,博士,教授,主要研究方向为计算机视觉、模式识别、信息安全.E-mail:bge@aust.edu.cn.

作者简介: 卢洋,硕士研究生,主要研究方向为计算机视觉、深度学习.E-mail:2022201255@aust.edu.cn.夏晨星,博士,副教授,主要研究方向为计算机视觉、模式识别、机器学习.E-mail:cxxia@aust.edu.cn. 官骏鸣,博士,副教授,主要研究方向为计算机视觉、模式识别.E-mail:105051@hsu.edu.cn.

引用本文:

葛斌, 卢洋, 夏晨星, 官骏鸣. 基于文本图像互学习的换衣行人重识别方法[J]. 模式识别与人工智能, 2024, 37(11): 960-973. GE Bin, LU Yang, XIA Chenxing, GUAN Junming. Clothes-Changing Person Re-identification Method Based on Text-Image Mutual Learning. Pattern Recognition and Artificial Intelligence, 2024, 37(11): 960-973.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/10.16451/j.cnki.issn1003-6059.202411002 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2024/V37/I11/960

[1] 罗浩,姜伟,范星,等.基于深度学习的行人重识别研究进展.自动化学报, 2019, 45(11): 2032-2049.
(LUO H, JIANG W, FAN X, et al. A Survey on Deep Learning Based Person Re-identification. Acta Automatica Sinica, 2019, 45(11): 2032-2049.)
[2] YU H X, WU A C, ZHENG W S. Unsupervised Person Re-identification by Deep Asymmetric Metric Embedding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(4): 956-973.
[3] LI M X, ZHU X T, GONG S G. Unsupervised Tracklet Person Re-identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(7): 1770-1782.
[4] 王鹏,宋晓宁,吴小俊,等.用于行人重识别的多类型特征网络.模式识别与人工智能, 2020, 33(10): 879-888.
(WANG P, SONG X N, WU X J, et al. Multi-type Features Network for Person Re-identification. Pattern Recognition and Artificial Intelligence, 2020, 33(10): 879-888.)
[5] 张磊,吴晓富,张索非,等.基于多分支协作的行人重识别网络.模式识别与人工智能, 2021, 34(9): 853-862.
(ZHANG L, WU X F, ZHANG S F, et al. Multi-branch Cooperative Network for Person Re-identification. Pattern Recognition and Artificial Intelligence, 2021, 34(9): 853-862.)
[6] SUN Y F, ZHENG L, YANG Y, et al. Beyond Part Models: Person Retrieval with Refined Part Pooling(and a Strong Convolutional Baseline) // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 501-518.
[7] HUANG Y, WU Q, XU J S, et al. Celebrities-ReID: A Benchmark for Clothes Variation in Long-Term Person Re-identification // Proc of the International Joint Conference on Neural Networks. Washington, USA: IEEE, 2019. DOI: 10.1109/IJCNN.2019.8851957.
[8] YANG Q Z, WU A C, ZHENG W S. Person Re-identification by Contour Sketch under Moderate Clothing Change. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(6): 2029-2046.
[9] WAN F B, WU Y, QIAN X L, et al. When Person Re-identification Meets Changing Clothes // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2020: 3620-3628.
[10] QIAN X L, WANG W X, ZHANG L, et al. Long-Term Cloth-Changing Person Re-identification // Proc of the Asian Conference on Computer Vision. Berlin, Germany: Springer, 2020: 71-88.
[11] FAN C, PENG Y J, CAO C S, et al. GaitPart: Temporal Part-Based Model for Gait Recognition // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 14213-14221.
[12] CHEN J X, ZHENG W S, YANG Q Z, et al. Deep Shape-Aware Person Re-identification for Overcoming Moderate Clothing Changes. IEEE Transactions on Multimedia, 2021, 24: 4285-4300.
[13] HONG P X, WU T, WU A C, et al. Fine-Grained Shape-Appea-rance Mutual Learning for Cloth-Changing Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2021: 10508-10517.
[14] SHU X J, LI G, WANG X, et al. Semantic-Guided Pixel Sampling for Cloth-Changing Person Re-identification. IEEE Signal Processing Letters, 2021, 28: 1365-1369.
[15] GAO Z, WEI S X, GUAN W L, et al. Identity-Guided Collaborative Learning for Cloth-Changing Person Reidentification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(5): 2819-2837.
[16] LI W J, HOU S H, ZHANG C J, et al. An In-Depth Exploration of Person Re-identification and Gait Recognition in Cloth-Changing Conditions // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 13824-13833.
[17] ZHANG G Q, LIU J, CHEN Y H, et al. Multi-biometric Unified Network for Cloth-Changing Person Re-identification // Proc of the IEEE International Conference on Multimedia and Expo. Washington, USA: IEEE, 2022. DOI: 10.1109/ICME52920.2022.9859702.
[18] LIU F Y, YE M, DU B. Dual Level Adaptive Weighting for Cloth-Changing Person Re-identification. IEEE Transactions on Image Processing, 2023, 32: 5075-5086.
[19] ZHENG Z D, YANG X D, YU Z D, et al. Joint Discriminative and Generative Learning for Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 2133-2142.
[20] HUANG Y, XU J S, WU Q, et al. Beyond Scalar Neuron: Adopting Vector-Neuron Capsules for Long-Term Person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(10): 3459-3471.
[21] GU X Q, CHANG H, MA B P, et al. Clothes-Changing Person Re-identification with RGB Modality Only // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 1050-1059.
[22] KWEON H J, CHO D. Cloth-Changing Person Re-identification with Noisy Patch Filtering. IEEE Signal Processing Letters, 2023,30: 334-338.
[23] ZHANG R J, FANG Y, SONG H X, et al. Specialized Re-ran-king: A Novel Retrieval-Verification Framework for Cloth Changing Person Re-identification. Pattern Recognition, 2023, 134. DOI: 10.1016/j.patcog.2022.109070.
[24] RADFORD A, KIM J W, HALLACY C, et al. Learning Transferable Visual Models from Natural Language Supervision // Proc of the 38th International Conference on Machine Learning. San Diego, USA: JMLR, 2021: 8748-8763.
[25] LI S Y, SUN L, LI Q L. CLIP-ReID: Exploiting Vision-Language Model for Image Re-identification without Concrete Text Labels. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(1): 1405-1413.
[26] HUANG Y, WU Q, XU J S, et al. Clothing Status Awareness for Long-Term Person Re-identification // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021, 11875-11884.
[27] JIN X, HE T Y, ZHENG K C, et al. Cloth-Changing Person Re-identification from a Single Image with Gait Prediction and Regula-rization // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 14258-14267.
[28] SHI W, LIU H, LIU M Y. IRANet: Identity-Relevance Aware Representation for Cloth-Changing Person Re-identification. Image and Vision Computing, 2022, 117. DOI: 10.1016/j.imavis.2021.104335.
[29] MU J Y, LI Y, LI J, et al. Learning Clothes-Irrelevant Cues for Clothes-Changing Person Re-identification[C/OL].[2024-09-07]. https://bmvc2022.mpi-inf.mpg.de/0337.pdf.
[30] GAO Z, WEI H W, GUAN W L, et al. Multigranular Visual-Semantic Embedding for Cloth-Changing Person Re-identification // Proc of the 30th ACM International Conference on Multimedia. New York, USA: ACM, 2022: 3703-3711.
[31] BANSAL V, FORESTI G L, MARTINEL N. Cloth-Changing Person Re-identification with Self-Attention // Proc of the IEEE/CVF Winter Conference on Applications of Computer Vision. Washington, USA: IEEE, 2022: 602-610.
[32] WANG Q Z, QIAN X L, FU Y W, et al. Co-attention Aligned Mutual Cross-Attention for Cloth-Changing Person Re-identification // Proc of the Asian Conference on Computer Vision. Berlin, Germany: Springer, 2022: 351-368.
[33] YANG Z W, ZHONG X, LIU H, et al. Attentive Decoupling Network for Cloth-Changing Re-identification // Proc of the IEEE International Conference on Multimedia and Expo. Washington, USA: IEEE, 2022. DOI: 10.1109/ICME52920.2022.9859851.
[34] LI X L, LU Y, LIU B, et al. Clothes-Invariant Feature Learning by Causal Intervention for Clothes-Changing Person Re-identification[C/OL].[2024-09-07]. https://arxiv.org/pdf/2305.06145.
[35] YANG Z W, LIN M, ZHONG X, et al. Good Is Bad: Causality Inspired Cloth-Debiasing for Cloth-Changing Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 1472-1481.
[36] YANG Z W, ZHONG X, ZHONG Z, et al. Win-Win by Competi-tion: Auxiliary-Free Cloth-Changing Person Re-identification. IEEE Transactions on Image Processing, 2023, 32: 2985-2999.
[37] CUI Z Y, ZHOU J H, PENG Y X, et al. DCR-ReID: Deep Component Reconstruction for Cloth-Changing Person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(8): 4415-4428.
[38] HAN K, GONG S G, HUANG Y, et al. Clothing-Change Feature Augmentation for Person Re-identification // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 22066-22075.
[39] WU Z Y, HU Z R, DING J W. Same-Clothes Person Re-identifi-cation with Dual-Stream Network. Multimedia Systems, 2024, 30(2). DOI: 10.1007/s00530-024-01269-0.
[40] TU H B, LIU C, PENG Y Y, et al. Clothing-Change Person Re-identification Based on Fusion of RGB Modality and Gait Features. Signal, Image and Video Processing, 2024, 18(3): 2367-2376.
[41] WANG G S, YUAN Y F, CHEN X, et al. Learning Discriminative Features with Multiple Granularities for Person Re-identification // Proc of the 26th ACM International Conference on Multimedia. New York, USA: ACM, 2018:274-282.
[42] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778.
[43] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely Connected Convolutional Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 2261-2269.