Abstract The generative zero shot recognition method is affected by redundant information and domain shifting while generating features, and thus its recognition accuracy is poor. To deal with the problem, a zero shot attribute recognition method based on de-redundant features and semantic relationship constraint is proposed. Firstly, the visual features are mapped to a new feature space, and the visual features are de-redundant via cross-correlation information. The redundant visual features are removed with the correlation of the categories preserved. The accuracy of zero shot recognition is improved due to the reduction of redundant information interference in the recognition process. Then, a knowledge transfer model is established using the semantic relationship between the seen and unseen classes, and the loss of semantic relationship is introduced to constrain the process of knowledge transfer. Consequently, the semantic relationship between the seen and unseen classes is reflected better by the visual features generated by the generator ,and the problem of domain shifting between them is alleviated as well. Finally, the cycle consistency structure is introduced to make the generated pseudo-features closer to the real features. Experiments on datasets show that the proposed method improves the accuracy of zero shot recognition tasks with better generalization performance.
Fund:National Natural Science Foundation of China(No.61462065,61763033)
Corresponding Authors:
ZHANG Guimei, Ph.D., professor. Her research interests include computer vision, image processing and pattern recognition.
About author:: LONG Bangyao, master student. His research interests include computer vision, image processing and pattern recognition. ZENG Jiexian, master, professor. His research interests include computer vision, image processing and pattern recognition. HUANG Junyang, master student. His research interests include computer vision, image processing and pattern recognition.
ZHANG Guimei,LONG Bangyao,ZENG Jiexian等. Zero-Shot Attribute Recognition Based on De-redundancy Features and Semantic Relationship Constraint[J]. , 2021, 34(9): 809-823.
[1] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet Classification with Deep Convolutional Neural Networks. Communications of the ACM, 2017, 60(6): 84-90. [2] LAROCHELLE H, ERHAN D, BENGIO Y. Zero-Data Learning of New Tasks // Proc of the 23rd National Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2008, II: 646-651. [3] ELHOSEINY M, YI K, ELFEJI M. CIZSL++: Creativity Inspired Generative Zero-Shot Learning[C/OL]. [2021-03-12]. https://arxiv.org/pdf/2101.00173v2.pdf. [4] ELHOSEINY M, SALEH B, ELGAMMAL A. Write a Classifier: Zero-Shot Learning Using Purely Textual Descriptions // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2013: 2584-2591. [5] ZHU Y Z, ELHOSEINY M, LIU B C, et al. A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 1004-1013. [6] ELHOSEINY M, ELFEKI M. Creativity Inspired Zero-Shot Learning // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 5782-5792. [7] CHEN Z, LI J J, LUO Y D, et al. CANZSL: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language // Proc of the IEEE Winter Conference on Applications of Computer Vision. Washington, USA: IEEE, 2020: 863-872. [8] LAMPERT C H, NICKISCH H, HARMELING S. Attribute-Based Classification for Zero-Shot Visual Object Categorization. IEEE Tran-sactions on Pattern Analysis and Machine Intelligence, 2014, 36(3): 453-465. [9] FROME A, CORRADO G S, SHLENS J, et al. DeViSE: A Deep Visual-Semantic Embedding Model // Proc of the 26th International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2013, II: 2121-2129. [10] ROMERA-PAREDES B, TORR P H S. An Embarrassingly Simple Approach to Zero-Shot Learning[C/OL]. [2021-03-12]. proc eedings.mlr.ress/romera-paredes15.pdf. [11] CHANGPINYO S, CHAO W L, GONG B Q, et al. Synthesized Classifiers for Zero-Shot Learning // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 5327-5336. [12] ZHANG L, XIANG T, GONG S G. Learning a Deep Embedding Model for Zero-Shot Learning // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 3010-3019. [13] 吴 凡,王 康.基于属性平衡正则化的深度零样本学习.计算机应用与软件, 2018, 35(10): 165-170. (WU F, WANG K. Deep Zero-Shot Learning Based on Attribute Balancing Regularization. Computer Applications and Software, 2018, 35(10): 165-170.) [14] 张鲁宁,左 信,刘建伟.零样本学习研究进展.自动化学报, 2020, 46(1): 1-23. (ZHANG L N, ZUO X, LIU J W. Research and Development on Zero-Shot Learning. Acta Automatica Sinica, 2020, 46(1): 1-23.) [15] YAO H T, MIN S B, ZHANG Y D, et al. Attribute-Induced Bias Eliminating for Transductive Zero-Shot Learning[C/OL]. [2021-03-12]. https://arxiv.org/pdf/2006.00412.pdf. [16] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Ge-nerative Adversarial Nets // Proc of the 27th International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2014, II: 2672-2680. [17] XIAN Y Q, LORENZ T, SCHIELE B, et al. Feature Generating Networks for Zero-Shot Learning // Proc of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 5542-5551. [18] ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[C/OL]. [2021-03-12]. https://arxiv.org/pdf/1701.07875.pdf. [19] VERMA V K, ARORA G, MISHRA A, et al. Generalized Zero-Shot Learning via Synthesized Examples // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 4281-4289. [20] FELIX R, KUMAR V B G, REID I, et al. Multi-modal Cycle-Consistent Generalized Zero Shot Learning // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 21-37. [21] SAEIYILDIZ M B, CINBIS R G. Gradient Matching Generative Networks for Zero-Shot Learning // Proc of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 2163-2173. [22] LI J J, JIN M M, LU K, et al. Leveraging the Invariant Side of Generative Zero Shot Learning // Proc of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 7394-7403. [23] SHERMIN T, TENG S W, SOHEL F, et al. Bidirectional Ma-pping Coupled GAN for Generalized Zero-Shot Learning[C/OL]. [2021-03-12]. https://arxiv.org/pdf/2012.15054v2.pdf. [24] CHANDHOK S, BALASUBRAMANIAN V N. Two-Level Adversarial Visual-Semantic Coupling for Generalized Zero-Shot Learning // Proc of the IEEE Winter Conference on Applications of Compu-ter Vision. Washington, USA: IEEE, 2021: 3100-3108. [25] XIANG H X, XIE C, ZENG T, et al. Multi-knowledge Fusion for New Feature Generation in Generalized Zero-Shot Learning[C/OL]. [2021-03-12]. https://arxiv.org/pdf/2102.11566.pdf. [26] LI B N, NIE X C, HAN C Y. DFS: A Diverse Feature Synthesis Model for Generalized Zero-Shot Learning[C/OL]. [2021-03-12]. https://arxiv.org/pdf/2103.10764.pdf. [27] ZHU J Y, PARK T, LSOLA P, et al. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 2242-2251. [28] WEI X, GONG B Q, LIU Z X, et al. Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect[C/OL]. [2021-03-12]. https://arxiv.org/pdf/1803.01541v1.pdf. [29] GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved Trai-ning of Wasserstein GANs // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2017: 5769-5779. [30] ALEMI A A, FISCHER I, DILLON J V, et al. Deep Variational Information Bottleneck[C/OL]. [2021-03-12]. https://openreview.net/pdf?id=HyxQzBceg. [31] WEN Y D, ZHANG K P, LI Z F, et al. A Discriminative Feature Learning Approach for Deep Face Recognition // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 499-515. [32] LILLO W E, LOH M H, HUI S, et al. On Solving Constrained Optimization Problems with Neural Networks: A Penalty Method Approach. IEEE Transactions on Neural Networks, 1993, 4(6): 931-940. [33] ZHENG Y, WU J H, QIN Y Q, et al. Zero-Shot Instance Segmentation // Proc of the IEEE Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2021: 2593-2602. [34] CHAO W L, CHANGPINYO S, GONG B Q, et al. An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 52-68.
2021, Vol. 34 
