Abstract To solve the problem of low retrieval accuracy and long training time in cross-modal retrieval algorithms, a cross-modal retrieval algorithm joining hashing feature and classifier learning (HFCL) is proposed. Uniform hash codes are utilized to describe different modal data with the same semantics. In the training stage, label information is utilized to study discriminative hash codes. And the kernel logistic regression is adopted to learn the hash function of each modal. In the testing stage, for any sample, the hash feature is generated by learned hash function, and another modal datum related to its semantics is retrieved from the database. Experiments on three public datasets verify the effectiveness of HFCL.
Fund:Supported by National Natural Science Foundation of China(No.61672265,U1836218), The 111 Project of Ministry of Education of China(No.B12018)
Corresponding Authors:
WU Xiaojun, Ph.D., professor. His research interests include artificial intelligence,pattern recognition and computer vision.
About author:: LIU Haoxin, master student. His research interests include cross-modal retrieval and hash learning; YU Jun, Ph.D. candidate. His research interests include multimodal retrieval and deep learning.
[1] LIU H, JI R R, WU Y J, et al. Cross-Modality Binary Code Learning via Fusion Similarity Hashing // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 6345-6353. [2] SONG J K, YANG Y, YANG Y, et al. Inter-Media Hashing for Large-Scale Retrieval from Heterogeneous Data Sources // Proc of the ACM SIGMOD International Conference on Management of Data. New York,USA: ACM, 2013: 785-796. [3] 张 鸿,吴 飞,庄越挺.基于特征子空间学习的跨媒体检索方法.模式识别与人工智能, 2008, 21(6): 739-745. (ZHANG H, WU F, ZHUANG Y T. Cross-Media Retrieval Method Based on Feature Subspace Learning. Pattern Recognition and Artificial Intelligence, 2008, 21(6): 739-745.) [4] SHEN X B, SHEN F M, SUN Q S, et al. Robust Cross-View Hashing for Multimedia Retrieval. IEEE Signal Processing Letters, 2016, 23(6): 893-897. [5] 庄 凌,王 超,周 峰,等.相关空间嵌入算法及其在图像检索中的应用.模式识别与人工智能, 2014, 27(4): 363-371. (ZHUANG L, WANG C, ZHOU F, et al. Correlation Space Embedding Algorithm and Its Application to Image Retrieval. Pattern Recognition and Artificial Intelligence, 2014, 27(4): 363-371.) [6] YU Y, WU X J, KITTLER J. Semi-supervised Hashing for Semi-Paired Cross-View Retrieval // Proc of the 24th International Conference on Pattern Recognition. Washington, USA: IEEE, 2018: 958-963. [7] ZHU X F, HUANG Z, SHEN H T, et al. Linear Cross-Modal Hashing for Efficient Multimedia Search // Proc of the 21st ACM International Conference on Multimedia. New York, USA: ACM, 2013: 143-152. [8] ZHENG F, TANG Y, SHAO L. Hetero-Manifold Regularisation for Cross-Modal Hashing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(5): 1059-1071. [9] DING G G, GUO Y C, ZHOU J. Collective Matrix Factorization Hashing for Multimodal Data // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 4321-4328. [10] ZHOU J L, DING G G, GUO Y C. Latent Semantic Sparse Hashing for Cross-Modal Similarity Search // Proc of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2014: 415-424. [11] BRONSTEIN M M, BRONSTEIN A M, MICHEL F, et al. Data Fusion through Cross-Modality Metric Learning Using Similarity-Sensitive Hashing // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2010: 3594-3601. [12] KUMAR S, UDUPA R. Learning Hash Functions for Cross-View Similarity Search // Proc of the 22nd International Joint Conference on Artificial Intelligence. New York, USA: ACM, 2011: 1360-1365. [13] ZHEN Y, YEUNG D Y. A Probabilistic Model for Multimodal Hash Function Learning // Proc of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2012: 940-948. [14] LIN Z J, DING G G, HU M Q, et al. Semantics-Preserving Hashing for Cross-View Retrieval // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 3864-3872. [15] ZHANG D Q, LI W J. Large-Scale Supervised Multimodal Hashing with Semantic Correlation Maximization // Proc of the 28th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2014: 2177-2183. [16] WRIGHT S J. Coordinate Descent Algorithms. Mathematical Programming, 2015, 151: 3-34. [17] PEREIRA J C, COVIELLO E, DOYLE G, et al. On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 36(3): 521-535. [18] RASHTCHIAN C, YOUNG P, HODOSH M, et al. Collecting Image Annotations Using Amazon′s Mechanical Turk // Proc of the NAACL HLT Workshop on Creating Speech and Language Data with Amazon′s Mechanical Turk. Stroudsburg, USA: ACL, 2010: 139-147.
2020, Vol. 33 
