Federated Domain Generalization Person Re-identification with Privacy Preserving
PENG Jinjia1, SONG Pengpeng1, WANG Huibing2
1. School of Cyber Security and Computer, Hebei University, Baoding 071002; 2. Information Science and Technology College, Dalian Maritime University, Dalian 116026
摘要 行人重识别旨在从不同的摄像头中识别目标行人的图像.由于不同场景之间存在域偏差,在一个场景中训练好的重识别模型常常无法直接应用在另一个场景,并且从摄像头收集的数据通常包含敏感的个人信息,而现有的大部分重识别方法通常需要训练数据的集中化,这可能会带来隐私泄露问题.因此,文中提出面向隐私保护的联邦域泛化行人重识别方法(Federated Domain Generalization Person Re-identification with Privacy Preserving, PFReID),在保护行人隐私的前提下,从独立的多个非共享数据域中学习泛化模型.使用频域空间插值的方法平滑各个客户端在数据集上的域偏差,增加样本的多样性,提高各客户端模型的泛化性能.在客户端构建双分支对齐学习网络,保证客户端局部模型和全局模型学习表示的一致性,用于客户端局部模型的更新.在多个公开行人数据集上的实验验证PFReID的性能较优.
Abstract：Person re-identification aims at recognizing images of target pedestrians in different cameras. The re-identification model trained in one scene cannot be directly applied in another scene, due to the domain bias between different scenes. The data collected from cameras often contains sensitive personal information. Most of the existing re-identification methods usually require centralization of training data, resulting in privacy leakage problems. Therefore, a method for federated domain generalization person re-identification with privacy preserving(PFReID) is proposed in this paper to learn a generalized model in a non-shared data domain with pedestrian privacy preserved. In PFReID, the frequency-domain spatial interpolation is introduced to smooth the domain deviation of each client on datasets, increase the diversity of samples and improve the generalization performance of client models. Moreover, a double-branch alignment learning network is designed for the update of the client-side local model by maximizing the consistency between the learned representation of the client-side local model and the learned representation of the global model. The superiority of PFReID is verified on public pedestrian datasets.
[1] ZHENG K C, LIU W, HE L X, et al. Group-Aware Label Transfer for Domain Adaptive Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 5306-5315. [2] 杨静,张灿龙,李志欣,等.集成空间注意力和姿态估计的遮挡行人再辨识.计算机研究与发展, 2022, 59(7): 1522-1532. (YANG J, ZHANG C L, LI Z X, et al. Integrated Spatial Attention and Pose Estimation for Occluded Person Re-identification. Journal of Computer Research and Development, 2022, 59(7): 1522-1532.) [3] 陆萍,董虎胜,钟珊,等.基于跨视角判别词典嵌入的行人再识别.计算机研究与发展, 2019, 56(11): 2424-2437. (LU P, DONG H S, ZHONG S, et al. Person Re-identification by Cross-View Discriminative Dictionary Learning with Metric Embe-dding. Journal of Computer Research and Development, 2019, 56(11): 2424-2437.) [4] QIAO F C, PENG X. Uncertainty-Guided Model Generalization to Unseen Domains // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 6786-6796. [5] JIN X, LAN C L, ZENG W J, et al. Style Normalization and Restitution for Domain Generalization and Adaptation. IEEE Transactions on Multimedia, 2021, 24: 3636-3651. [6] LI L, GAO K, CAO J, et al. Progressive Domain Expansion Network for Single Domain Generalization // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 224-233. [7] WU G L, GONG S G. Decentralised Learning from Independent Multi-domain Labels for Person Re-identification. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(4): 2898-2906. [8] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of Machine Leaning Research, 2017, 54: 1273-1282. [9] JIAO B L, LIU L Q, GAO L Y, et al. Dynamically Transformed Instance Normalization Network for Generalizable Person Re-identification // Proc of the 17th European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 285-301. [10] ZHANG L, LIU Z P, ZHANG W S, et al. Style Uncertainty Based Self-Paced Meta Learning for Generalizable Person Re-identification. IEEE Transactions on Image Processing, 2023, 32: 2107-2119. [11] ZHAO J J, ZHAO Y F, CHEN X W, et al. Revisiting Stochastic Learning for Generalizable Person Re-identification // Proc of the 30th ACM International Conference on Multimedia. New York, USA: ACM, 2022: 1758-1768. [12] ZHAO Y Y, ZHONG Z, YANG F X, et al. Learning to Generalize Unseen Domains via Memory-Based Multi-source Meta-Learning for Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 6273-6282. [13] NI H, SONG J K, LUO X P, et al. Meta Distribution Alignment for Generalizable Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 2477-2486. [14] ZHOU K Y, YANG Y X, CAVALLARO A, et al. Learning Generalisable Omni-Scale Representations for Person Re-identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9): 5056-5069. [15] ZHANG P Y, DOU H Z, YU Y L, et al. Adaptive Cross-Domain Learning for Generalizable Person Re-identification // Proc of the 17th European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 215-232. [16] GONG T T, CHEN K X, ZHANG L Y, et al. Debiased Contrastive Curriculum Learning for Progressive Generalizable Person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 2023. DOI: 10.1109/TCSVT.2023.3262832. [17] LI L, FAN Y X, TSE M, et al. A Review of Applications in Fe-derated Learning. Computers and Industrial Engineering, 2020, 149. DOI: 10.1016/j.cie.2020.106854. [18] ZHUANG W M, GAN X, WEN Y G, et al. Optimizing Perfor-mance of Federated Person Re-identification: Benchmarking and Analysis. ACM Transactions on Multimedia Computing, Communi-cations, and Applications, 2023, 19(1s). DOI: 10.1145/3531013. [19] YANG F X, ZHONG Z, LUO Z M, et al. Federated and Genera-lized Person Re-identification through Domain and Feature Hallucinating[C/OL].[2023-03-22]. https://arxiv.org/pdf/2203.02689.pdf. [20] SUN S T, WU G L, GONG S G. Decentralised Person Re-identification with Selective Knowledge Aggregation[C/OL]. [2023-03-22]. https://arxiv.org/pdf/2110.11384v1.pdf. [21] ZHONG Z, ZHENG L, ZHENG Z D, et al. Camstyle: A Novel Data Augmentation Method for Person Re-identification. IEEE Transactions on Image Processing, 2019, 28(3): 1176-1190. [22] WANG H H, WU X D, HUANG Z Y, et al. High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 8681-8691. [23] ZHENG L, SHEN L Y, TIAN L, et al. Scalable Person Re-identification: A Benchmark // Proc of the IEEE International Confe-rence on Computer Vision. Washington, USA: IEEE, 2015: 1116-1124. [24] ZHENG Z D, ZHENG L, YANG Y. Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 3774-3782. [25] WEI L H, ZHANG S L, GAO W, et al. Person Transfer GAN to Bridge Domain Gap for Person Re-identification // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 79-88. [26] LI W, ZHAO R, XIAO T, et al. DeepReID: Deep Filter Pairing Neural Network for Person Re-identification // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 152-159. [27] GRAY D, TAO H. Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2008: 262-275. [28] ZHENG W S, GONG S G, XIANG T. Associating Groups of Peo-ple[C/OL]. [2023-03-22]. http://www.eecs.qmul.ac.uk/~sgg/papers/ZhengGongXiang_BMVC09.pdf. [29] PASZKE A, GROSS S, CHINTALA S, et al. Automatic Differentiation in Pytorch[C/OL].[2023-03-22]. https://openreview.net/pdf?id=BJJsrmfCZ. [30] LI D, YANG Y X, SONG Y Z, et al. Learning to Generalize: Meta-Learning for Domain Generalization. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1): 3490-3497. [31] SONG J F, YANG Y X, SONG Y Z, et al. Generalizable Person Re-identification by Domain-Invariant Mapping Network // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 719-728. [32] PENG P X, XIANG T, WANG Y W, et al. Unsupervised Cross-dataset Transfer Learning for Person Re-identification // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1306-1315. [33] YANG Q Z, YU H X, WU A C, et al. Patch-Based Discriminative Feature Learning for Unsupervised Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Re-cognition. Washington, USA: IEEE, 2019: 3628-3637. [34] WANG J Y, ZHU X T, GONG S G, et al. Transferable Joint Attribute-Identity Deep Learning for Unsupervised Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 2275-2284. [35] HU J L, LU J W, TAN Y P. Deep Transfer Metric Learning // Proc of the IEEE Conference on Computer Vision and Pattern Re-cognition. Washington, USA: IEEE, 2015: 325-333. [36] BAK S, CARR P, LALONDE J F. Domain Adaptation through Synthesis for Unsupervised Person Re-identification // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 193-209. [37] SU C, ZHANG S L, XING J L, et al. Deep Attributes Driven Multi-camera Person Re-identification // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 475-491. [38] SHANKAR S, PIRATLA V, CHAKRABARTI S, et al. Generalizing across Domains via Cross-Gradient Training[C/OL].[2023-03-22]. https://arxiv.org/pdf/1804.10745.pdf. [39] XU X, LIU W, WANG Z, et al. Towards Generalizable Person Re-identification with a Bi-stream Generative Model[C/OL].[2023-03-22]. https://arxiv.org/pdf/2206.09362v1.pdf. [40] FAN H H, ZHENG L, YAN C G, et al. Unsupervised Personre-Identification: Clustering and Fine-Tuning.ACM Transactions on Multimedia Computing, Communications, and Applications, 2018, 14(4). DOI: 10.1145/3243316. [41] DENG W J, ZHENG L, YE Q X, et al. Image-Image Domain Adap-tation with Preserved Self-Similarity and Domain Dissimilarity for Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 994-1003. [42] ZHONG Z, ZHENG L, LI S Z, et al. Generalizing a Person Retrieval Model Hetero-and Homogeneously // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 176-192. [43] LIN Y T, DONG X Y, ZHENG L, et al. A Bottom-Up Clustering Approach to Unsupervised Person Re-identification. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 8738-8745. [44] ZHUANG W M, WEN Y G, ZHANG S. Joint Optimization in Edge-Cloud Continuum for Federated Unsupervised Person Re-identification // Proc of the 29th ACM International Conference on Multimedia. New York, USA: ACM, 2021: 433-441. [45] LIAO S C, SHAO L. Interpretable and Generalizable Person Re-identification with Query-Adaptive Convolution and Temporal Lif-ting // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 456-474. [46] CHOI S, KIM T, JEONG M, et al. Meta Batch-Instance Normalization for Generalizable Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 3424-3434. [47] JIN X, LAN C L, ZENG W J, et al. Style Normalization and Restitution for Generalizable Person Re-identification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 3140-3149. [48] JOO H T, KIM K J. Visualization of Deep Reinforcement Learning Using Grad-CAM: How AI Plays Atari Games? // Proc of the IEEE Conference on Games. Washington, USA: IEEE, 2019. DOI: 10.1109/CIG.2019.8847950.
2023, Vol. 36 
