Abstract There are plenty of studies on improving the performance of pedestrian attribute recognition in video surveillance scenarios by mining the positive correlations between attributes. However, the research on negative correlations is far from enough. In this paper, a deep learning based multi-stage pedestrian attribute recognition method is proposed to simultaneously explore the positive and negative correlations between attributes. In the first stage, the loss value and the accuracy of each attribute are calculated during training. In the second stage, a new network branch is designed for the attributes with larger average loss and lower average accuracy, while other attributes remain on the original branch. All attributes are predicted by these two branches jointly. In the third stage, two new network branches with same structure as the second stage are designed to optimize the parameters and improve the performance during attribute recognition. Moreover, an improved loss function increasing the distance between positive and negative samples is proposed, and it is applied in all training stages to further improve the performance. Experiments on datasets RAP and PETA validate the promising performance of the proposed method.
Fund:Supported by National Natural Science Foundation of China(No.61671018,61472002,61502006)
About author:: (ZHENG Shaofei, master student. His research interests include computer vision and machine learning.) (TANG Jin(Corresponding author), Ph.D., professor. His research interests include computer vision and big data processing.) (LUO Bin, Ph.D., professor. His research interests include pattern recognition and graph matching theory.) (WANG Xiao, Ph.D. candidate. His research interests include computer vision and pattern recognition.) (WANG Wenzhong, Ph.D., lecturer. His research interests include computer vision and machine learning.)
[1] KUMAR N, BERG A, BELHUMEUR P N, et al. Describable Vi-sual Attributes for Face Verification and Image Search. IEEE Tran-sactions on Pattern Analysis and Machine Intelligence, 2011, 33(10): 1962-1977. [2] LIU Z W, LUO P, WANG X G, et al. Deep Learning Face Attri-butes in the Wild // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 3730-3738. [3] LUO P, WANG X G, TANG X O. A Deep Sum-Product Architecture for Robust Facial Attributes Analysis // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2013: 2864-2871. [4] LAYNE R, HOSPEDALES T M, GONG S G. Person Re-identification by Attributes // Proc of the 23rd British Machine Vision Confe-rence. Berlin, Germany: Springer, 2012. DOI: 10.5244/C.26.24. [5] ZHU J Q, LIAO S C, YI D, et al. Multi-label CNN Based Pede-strian Attribute Learning for Soft Biometrics // Proc of the International Conference on Biometrics. Washington, USA: IEEE, 2015: 535-540. [6] REID D A, NIXON M S, STEVENAGE S V. Soft Biometrics; Human Identification Using Comparative Descriptions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(6): 1216-1228. [7] DENG Y B, LUO P, LOY C C, et al. Pedestrian Attribute Recognition at Far Distance // Proc of the 22nd ACM International Confe-rence on Multimedia. New York, USA: ACM, 2014: 789-792. [8] JAHA E S, NIXON M S. Soft Biometrics for Subject Identification Using Clothing Attributes // Proc of the IEEE International Joint Conference on Biometrics. Washington, USA: IEEE, 2014. DOI: 10.1109/BTAS.2014.6996278. [9] LIU C X, GONG S G, LOY C C, et al. Person Re-identification: What Features Are Important? // Proc of the European Conference on Computer Vision. Washington, USA: Springer, 2012: 391-401. [10] CHEN H Z, GALLAGHER A, GIROD B. Describing Clothing by Semantic Attributes // Proc of the European Conference on Computer Vision. Washington, USA: Springer, 2012: 609-623. [11] DENG Y B, LUO P, LOY C C, et al. Learning to Recognize Pedestrian Attribute[C/OL]. [2018-08-25]. https://arxiv.org/pdf/1501.00901.pdf. [12] SHI Z Y, HOSPEDALES T M, XIANG T. Transferring a Semantic Representation for Person Re-identification and Search // Proc of the IEEE International Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 4184-4193. [13] LI D W, CHEN X T, HUANG K Q. Multi-attribute Learning for Pedestrian Attribute Recognition in Surveillance Scenarios // Proc of the 3rd IARP Asian Conference on Pattern Recognition. Washington, USA: IEEE, 2015: 111-115. [14] DONG Q, GONG S G, ZHU X T. Class Rectification Hard Mining for Imbalanced Deep Learning // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 1851-1860. [15] DONG Q, GONG S G, ZHU X T. Multi-task Curriculum Transfer Deep Learning of Clothing Attributes // Proc of the IEEE Winter Conference on Applications of Computer Vision. Washington, USA: IEEE, 2017: 520-529. [16] PENG P X, TIAN Y H, XIANG T, et al. Joint Semantic and Latent Attribute Modelling for Cross-Class Transfer Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(7): 1625-1638. [17] ZHU J Q, LIAO S C, LEI Z, et al. Multi-label Convolutional Neural Network Based Pedestrian Attribute Classification. Image and Vision Computing, 2017, 58: 224-229. [18] WANG J Y, ZHU X T, GONG S G. Video Semantic Clustering with Sparse and Incomplete Tags // Proc of the 30th AAAI Confe-rence on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2016: 3618-3624. [19] WANG J Y, ZHU X T, GONG S G. Discovering Visual Concept Structure with Sparse and Incomplete Tags. Artificial Intelligence, 2017, 250: 16-36. [20] LIU X H, ZHAO H Y, TIAN M Q, et al. HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 350-359. [21] YU K, LENG B, ZHANG Z, et al. Weakly-Supervised Learning of Mid-level Features for Pedestrian Attribute Recognition and Localization[C/OL]. [2018-08-25]. https://arxiv.org/pdf/1611.05603.pdf. [22] LI Y N, HUANG C, LOY C C, et al. Human Attribute Recognition by Deep Hierarchical Contexts // Proc of the European Confe-rence on Computer Vision. Washington, USA: Springer, 2016: 684-700. [23] GKIOXARI G, GIRSHICK R, MALIK J. Contextual Action Recognition with R*CNN // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 1080-1088. [24] ZHANG N, PALURI M, RANZATO M A, et al. PANDA: Pose Aligned Networks for Deep Attribute Modeling // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 1637-1644. [25] LI D W, ZHANG Z, CHEN X T, et al. A Richly Annotated Dataset for Pedestrian Attribute Recognition[C/OL]. [2018-08-25]. https://arxiv.org/pdf/1603.07054.pdf. [26] FABBRI M, CALDERARA S, CUCCHIARA R. Generative Adversarial Models for People Attribute Recognition in Surveillance[C/OL]. [2018-08-25]. https://arxiv.org/pdf/1707.02240.pdf. [27] WANG J Y, ZHU X T, GONG S G, et al. Attribute Recognition by Joint Recurrent Learning of Context and Correlation[C/OL]. [2018-08-25]. https://arxiv.org/pdf/1709.08553.pdf. [28] SUDOWE P, SPITZER H, LEIBE B. Person Attribute Recognition with a Jointly-Trained Holistic CNN Model // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 87-95. [29] SUDOWE P, LEIBE B. Patchit: Self-supervised Network Weight Initialization for Fine-Grained Recognition // Proc of the British Machine Vision Conference. Berlin, Germany: Springer, 2016: 75.1-75.12. [30] HAND E M, CHELLAPPA R. Attributes for Improved Attributes: A Multi-task Network for Attribute Classification[C/OL]. [2018-08-25]. https://arxiv.org/pdf/1604.07360.pdf. [31] SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition[C/OL]. [2018-08-25]. https://arxiv.org/pdf/1409.1556.pdf. [32] YOSINSKI J, CLUNE J, BENGIO Y, et al. How Transferable Are Features in Deep Neural Networks? [C/OL]. [2018-08-25]. https://arxiv.org/pdf/1411.1792.pdf. [33] LI Y, LIN G S, ZHUANG B H, et al. Sequential Person Recognition in Photo Albums with a Recurrent Network // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 1338-1346. [34] LIU F, XIANG T, HOSPEDALES T M, et al. Semantic Regularisation for Recurrent Image Annotation // Proc of the IEEE International Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 2872-2880. [35] LAYNE R, HOSPEDALES T M, GONG S G. Attributes-Based Re-identification // GONG S G, CRISTANI M, YAN S C, et al., eds. Person Re-identification. Berlin, Germany: Springer, 2014: 93-117.
2018, Vol. 31 
