稠密向量实体检索模型的二值化提速压缩

doi:10.16451/j.cnki.issn1003-6059.202301005

Abstract
Figure/Table
References
Related Citation (15)

Download: PDF (873 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract In entity retrieval tasks, dense vector entity retrieval models are utilized to efficiently filter candidate entities related to a query from a large-scale entity base.However, the existing dense vector retrieval models engender low real-time computation efficiency and large required storage space due to the high dimension of entity vectors. In this paper,it is found that these entity vectors contain a large amount of redundant information through experiments. Most entity vectors are distributed in non-overlapping quadrants and quadrants containing entities with similar semantics are also closer to each other.Thus, a binary entity retrieval method is proposed to compress entity vectors and accelerate similarity calculations.Specifically, the sign function is employed to binary-compress high-dimensional dense floating-point vectors, and Hamming distance is exploited to speed up the retrieval.The reason that the proposed method can guarantee the retrieval performance is theoretically analyzed.The correctness of the theory is verified through qualitative and quantitative analysis experiments, and a method for improving binary retrieval performance based on random dimension increase and rotation is provided.

Key words： Entity Retrieval Embedding Representation Dense Vector Retrieval Approximate Nearest Neighbor Retrieval Product Quantization

Received: 20 September 2022

ZTFLH:

TP391

Fund:National Natural Science Foundation of China(No.61902381,62006218), Youth Innovation Promotion Association Member Project of Chinese Academy of Sciences(No.20144310,2021100), Young Elite Scientist Sponsorship Program by China Association for Science and Technology(No.YESS20200121), Innovation Project of Institute of Computing Technology of Chinese Academy of Sciences(No.E261090)

Corresponding Authors: FAN Yixing, Ph.D., associate professor. His research interests include data mining and information retrieval.

About author:: WANG Yuanzheng, Ph.D. candidate. His research interests include information retrieval and natural language processing.CHEN Wei, Ph.D., professor. Her research interests include machine learning.ZHANG Ruqing, Ph.D., assistant profe-ssor. Her research interests include natural language processing.GUO Jiafeng, Ph.D., professor. His research interests include data mining and information retrieval.

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	WANG Yuanzheng
	FAN Yixing
	CHEN Wei
	ZHANG Ruqing
	GUO Jiafeng

Cite this article:

WANG Yuanzheng,FAN Yixing,CHEN Wei等. Binary Acceleration and Compression for Dense Vector Entity Retrieval Models[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(1): 60-69.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.202301005 OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2023/V36/I1/60

[1] HAN X P, SUN L, ZHAO J.Collective Entity Linking in Web Text: A Graph-Based Method // Proc of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2011: 765-774.
[2] XIONG C Y, LIU Z Z, CALLAN J, et al. JointSem: Combining Query Entity Linking and Entity Based Document Ranking // Proc of the ACM Conference on Information and Knowledge Management. New York, USA: ACM, 2017: 2391-2394.
[3] NOORALAHZADEH F, ÓVRELID L.SIRIUS-LTG: An Entity Lin-king Approach to Fact Extraction and Verification // Proc of the 1st Workshop on Fact Extraction and VERification. Stroudsburg, USA: ACL, 2018: 119-123.
[4] FERRUCCI D A.Introduction to "This Is Watson". IBM Journal of Research and Development, 2012, 56(3/4). DOI: 10.1147/JRD.2012.2184356.
[5] CHEN D Q, FISCH A, WESTON J, et al. Reading Wikipedia to Answer Open-Domain Questions // Proc of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2017: 1870-1879.
[6] LEWIS P, PEREZ E, PIKTUS A, et al.Retrieval-Augmented Ge-neration for Knowledge-Intensive NLP Tasks // Proc of the 34th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2020: 9459-9474.
[7] ROLLER S, DINAN E, GOYAL N, et al. Recipes for Building an Open-Domain Chatbot // Proc of the 16th Conference of the European Chapter of the Association for Computational Linguistics(Main Volume). Stroudsburg, USA: ACL, 2021: 300-325.
[8] XIONG C Y, CALLAN J, LIU T Y.Word-Entity Duet Representations for Document Ranking // Proc of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2017: 763-772.
[9] HASIBI F, BALOG K, BRATSBERG S E.Exploiting Entity Lin-king in Queries for Entity Retrieval // Proc of the ACM International Conference on the Theory of Information Retrieval. New York, USA: ACM, 2016: 209-218.
[10] BALOG K, RAMAMPIARO H, TAKHIROV N, et al. Multi-step Classification Approaches to Cumulative Citation Recommendation // Proc of the 10th Conference on Open Research Areas in Information Retrieval. New York, USA: ACM, 2013: 121-128.
[11] REINANDA R, MEIJ E, DE RIJKE M.Mining, Ranking and Recommending Entity Aspects // Proc of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2015: 263-272.
[12] YANG Y, IRSOY O, RAHMAN K S.Collective Entity Disambiguation with Structured Gradient Tree Boosting // Proc of the Confe-rence of the North American Chapter of the Association for Computational Linguistics(Human Language Technologies). Stroudsburg, USA: ACL, 2018: 777-786.
[13] WU L, PETRONI F, JOSIFOSKI M, et al. Scalable Zero-Shot Entity Linking with Dense Entity Retrieval // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2020: 6397-6407.
[14] YAMADA I, ASAI A, SHINDO H, et al. LUKE: Deep Contextualized Entity Representations with Entity-Aware Self-Attention // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2020: 6442-6454.
[15] WANG Y Z, CHENG X Q, FAN Y X, et al. MGAD: Learning Descriptional Representation Distilled from Distributional Semantics for Unseen Entities // Proc of the 31st International Joint Confe-rence on Artificial Intelligence. San Francisco, USA: IJCAI, 2022: 4404-4410.
[16] JÉGOU H, DOUZE M, SCHMID C. Product Quantization for Nearest Neighbor Search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(1): 117-128.
[17] HOFFART J, YOSEF M A, BORDINO I, et al. Robust Disambi-guation of Named Entities in Text // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2011: 782-792.
[18] VAN HULST J M, HASIBI F, DERCKSEN K, et al. REL: An Entity Linker Standing on the Shoulders of Giants // Proc of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM, 2020: 2197-2200.
[19] DELIN J, CHANG M W, LEE K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics(Human Language Technologies). Stroudsburg, USA: ACL, 2019: 4171-4186.
[20] YAMADA I, ASAI A, HAJISHIRZI H.Efficient Passage Retrieval with Hashing for Open-Domain Question Answering // Proc of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg, USA: ACL, 2021: 979-986.
[21] JEGOU H, DOUZE M, SCHMID C.Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search // Proc of the European Conference on Computer Vision. Berlin, Ger-many: Springer, 2008: 304-317.
[22] GONG Y C, LAZEBNIK S, GORDO A, et al. Iterative Quantization: A Procrustean Approach to Learning Binary Codes for Large-Scale Image Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12): 2916-2929.