二值表示学习及其应用

doi:10.16451/j.cnki.issn1003-6059.201801002

摘要
图/表
参考文献(0)
相关文章 (0)

全文: PDF (1022 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要

随着互联网等信息技术的飞速发展,视觉数据呈现爆炸式增长趋势.如何从视觉数据中高效挖掘信息,已成为大数据时代的重要研究课题.二值表示在存储、传输和匹配上表现出高效性,在多项视觉应用中取得初步成效.文中从实际应用出发,对面向视觉搜索和视觉识别的二值表示学习进行总结分析.在方法层面上,主要从图像哈希和视频哈希两个角度进行阐释.在应用层面上,讨论二值表示学习在人脸分析、图像分类、图像匹配和视觉跟踪任务上的成功应用.最后,对二值表示学习的发展趋势进行展望.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	鲁继文
	段岳圻
	陈志祥
	周杰

关键词 ：二值表示, 特征学习, 视觉识别, 视觉搜索

Abstract：

With the rapid development of internet and other information technologies, the amount of visual data grows explosively. It is an essential research topic for efficient visual data mining in the big data era. In this paper, the technology of binary representation learning and its applications are reviewed. Binary representation shows efficiency in storage, transmission and matching, and it is successfully applied to several visual analysis tasks. The applications on visual search and recognition are summarized, and the future trend of binary representation learning is pointed out. From the aspect of methodology, the image-based hashing and the video-based hashing are illustrated, respectively. From the aspect of applications, the successful applications of binary representation learning on varying tasks are discussed including facial analysis, image categorization, image matching and visual tracking. Finally, the future trend of binary representation learning is introduced.

Key words： Binary Representation Feature Learning Visual Recognition Visual Search

收稿日期: 2017-09-26

基金资助:

国家重点研发计划项目(No.2016YFB1001001)、国家重点基础研究发展计划(973计划)(No.2014CB349304)、国家自然科学基金项目(No.61672306,61572271,61527808,61373074, 61373090)、国家青年千人计划、教育部博士点基金项目(No.20120002110033)、NSFC-深圳机器人基础研究中心项目(No.JCYJ20170412170602564)、清华大学自主科研计划、博士后创新人才支持计划项目(No.BX201700137)资助

作者简介: 鲁继文,博士,副教授,主要研究方向为计算机视觉、模式识别、机器学习.E-mail:lujiwen@tsinghua.edu.cn.段岳圻,博士研究生,主要研究方向为计算机视觉.E-mail:duanyq14@mails.tsinghua.edu.cn.陈志祥,博士,助理研究员,主要研究方向为计算机视觉.E-mail:zxchen@mail.tsinghua.edu.cn.周杰(通讯作者),博士,教授,主要研究方向为计算机视觉、模式识别、图像处理.E-mail:jzhou@tsinghua.edu.cn.

引用本文:

鲁继文, 段岳圻, 陈志祥, 周杰. 二值表示学习及其应用[J]. 模式识别与人工智能, 2018, 31(1): 12-22. LU Jiwen, DUAN Yueqi, CHEN Zhixiang, ZHOU Jie. Binary Representation Learning and Its Applications. , 2018, 31(1): 12-22.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/10.16451/j.cnki.issn1003-6059.201801002 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2018/V31/I1/12

[1] GIRSHICK R. Fast R-CNN // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 1440-1448.
[2] HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778.
[3] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-Speed Tracking with Kernelized Correlation Filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583-596.
[4] WANG J D, ZHANG T, SONG J K, et al. A Survey on Learning to Hash. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. DOI: 10.1109/TPAMI.2017.2699960.
[5] LU J W, LIONG V E, ZHOU X Z, et al. Learning Compact Binary Face Descriptor for Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(10): 2041-2056.
[6] LIN K, LU J W, CHEN C S, et al. Learning Compact Binary Descriptors with Unsupervised Deep Neural Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Wa-
shington, USA: IEEE, 2016: 1183-1192.
[7] DUAN Y Q, LU J W, WANG Z W, et al. Learning Deep Binary Descriptor with Multi-quantization // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 4857-4866.
[8] PARKHI O M, SIMONYAN K, VEDALDI A, et al. A Compact and Discriminative Face Track Descriptor // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 1693-1700.
[9] GONG Y C, LAZEBNIK S, GORDO A, et al. Iterative Quantization: A Procrustean Approach to Learning Binary Codes for Large-Scale Image Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12): 2916-2929.
[10] JOLLIFFE I T. Principal Component Analysis and Factor Analysis // JOLLIFFE I T, eds. Principal Component Analysis. New York, USA: Springer, 1986: 115-128.
[11] HARTIGAN J A, MANCHEK A W. Algorithm AS 136: A K-means Clustering Algorithm. Journal of the Royal Statistical Society(Applied Statistics), 1979, 28(1): 100-108.
[12] CAJO J F, TER B. Canonical Correspondence Analysis: A New Eigenvector Technique for Multivariate Direct Gradient Analysis. Ecology, 1986, 67(5): 1167-1179.
[13] HE K M, WEN F, SUN J. K-means Hashing: An Affinity-Preserving Quantization Method for Learning Binary Compact Codes // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2013: 2938-2945.
[14] WEISS Y, TORRALBA A, FERGUS R. Spectral Hashing // KOLLER D, SCHUURMANS D, BENGIO Y, et al., eds. Advances in Neural Information Processing Systems 21. Cambridge, USA: The MIT Press, 2009: 1753-1760.
[15] MATSUSHITA Y, WADA T. Principal Component Hashing: An Accelerated Approximate Nearest Neighbor Search // Proc of the Pacific-Rim Symposium on Image and Video Technology. Berlin, Germany: Springer, 2009: 374-385.
[16] HE J F, CHANG S F, RADHAKRISHNAN R, et al. Compact Hashing with Joint Optimization of Search Accuracy and Time // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2011: 753-760.
[17] ZHU X F, HUANG Z, CHENG H, et al. Sparse Hashing for Fast Multimedia Search. ACM Transactions on Information Systems(TOIS), 2013. DOI: 10.1145/2457465.2457469.
[18] LI P, WANG M, CHENG J, et al. Spectral Hashing with Semantically Consistent Graph for Image Indexing. IEEE Transactions on Multimedia, 2013, 15(1): 141-152.
[19] HE J F, LIU W, CHANG S F. Scalable Similarity Search with Optimized Kernel Hashing // Proc of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2010: 1129-1138.
[20] YANG Y, SHEN F M, SHEN H T, et al. Robust Discrete Spectral Hashing for Large-Scale Image Semantic Indexing. IEEE Transactions on Big Data, 2015, 1(4): 162-171.
[21] WANG J, KUMAR S, CHANG S F. Sequential Projection Lear-ning for Hashing with Compact Codes // Proc of the 27th International Conference on Machine Learning. West Norriton, USA: Omnipress, 2010: 1127-1134.
[22] HAO Y B, MU T T, HONG R C, et al. Stochastic Multiview Ha-shing for Large-Scale Near-Duplicate Video Retrieval. IEEE Tran-sactions on Multimedia, 2017, 19(1): 1-14.
[23] DONG Z, JIA S, WU T F, et al. Face Video Retrieval via Deep Learning of Binary Hash Representations[C/OL]. [2017-11-25]. http://www.ams.stonybrook.edu/~jias/AAAI16.pdf.
[24] LIU W, WANG J, KUMAR S, et al. Hashing with Graphs[C/OL]. [2017-11-25]. http://www.icml-2011.org/papers/6_icmlpaper.pdf.
[25] LIU W, MU C, KUMAR S, et al. Discrete Graph Hashing[C/OL]. [2017-11-25]. http://www.ee.columbia.edu/~wliu/NIPS14_dgh.pdf.
[26] LIONG V E, LU J W, WANG G, et al. Deep Hashing for Compact Binary Codes Learning // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 2475-2483.
[27] KULIS B, DARRELL T. Learning to Hash with Binary Reconstructive Embeddings // BENGIO Y, SCHUURMANS D, LAFFERLY J D, eds. Advance in Neural Information Processing Systems 22. Cambridge, USA: The MIT Press, 2009: 1042-1050.
[28] SHEN F M, SHEN C H, LIU W, et al. Supervised Discrete Ha-shing // Proc of the International Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 37-45.
[29] CHEN Z X, LU J W, FENG J J, et al. Nonlinear Discrete Ha-shing. IEEE Transactions on Multimedia, 2017, 19(1): 123-135.
[30] CHEN Z X, LU J W, FENG J J, et al. Nonlinear Sparse Hashing. IEEE Transactions on Multimedia, 2017, 19(9): 1996-2009.
[31] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet Cla-ssification with Deep Convolutional Neural Networks // PEREIRA F, BURGES C J C, BOTTOU L, et al., eds. Advance in Neural Information Processing Systems 25. Cambridge, USA: The MIT Press, 2012: 1097-1105.
[32] SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition[J/OL]. [2017-11-25]. https://arxiv.org/pdf/1409.1556.pdf.
[33] XIA R K, PAN Y, LAI H J, et al. Supervised Hashing for Image Retrieval via Image Representation Learning // Proc of the 28th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2014: 2156-2162.
[34] ZHU H, LONG M S, WANG J M, et al. Deep Hashing Network for Efficient Similarity Retrieval // Proc of the 30th AAAI Confe-rence on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2016: 2415-2421.
[35] CAO Y, LONG M S, WANG J M. Correlation Hashing Network for Efficient Cross-Modal Retrieval[C/OL]. [2017-11-25]. https://arxiv.org/pdf/1602.06697.pdf.
[36] CAO Y, LONG M S, WANG J M, et al. Deep Visual-Semantic Hashing for Cross-Modal Retrieval // Proc of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2016: 1445-1454.
[37] CAO Y, LONG M S, WANG J M, et al. Collective Deep Quantization for Efficient Cross-Modal Retrieval // Proc of the AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2017: 3974-3980.
[38] LIU H M, WANG R P, SHAN S G, et al. Deep Supervised Ha-shing for Fast Image Retrieval // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 2064-2072.
[39] SONG J K, YANG Y, HUANG Z, et al. Effective Multiple Feature Hashing for Large-Scale Near-Duplicate Video Retrieval. IEEE Transactions on Multimedia, 2013, 15(8): 1997-2008.
[40] LAI H J, PAN Y, LIU Y, et al. Simultaneous Feature Learning and Hash Coding with Deep Neural Networks // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 3270-3278.
[41] DOUZE M, JEGOU H, SCHMID C. An Image-Based Approach to Video Copy Detection with Spatio-Temporal Post-Filtering. IEEE Transactions on Multimedia, 2010, 12(4): 257-266.
[42] COSKUN B, SANKUR B, MEMON N. Spatio-Temporal Transform Based Video Hashing. IEEE Transactions on Multimedia, 2006, 8(6): 1190-1208.
[43] YE G N, LIU D, WANG J, et al. Large-Scale Video Hashing via Structure Learning // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2013: 2272-2279.
[44] CAO L L, LI Z G, MU Y D, et al. Submodular Video Hashing: A Unified Framework towards Video Pooling and Indexing // Proc of the 20th ACM International Conference on Multimedia. New York, USA: ACM, 2012: 299-308.
[45] YU L T, HUANG Z, CAO J W, et al. Scalable Video Event Retrieval by Visual State Binary Embedding. IEEE Transactions on Multimedia, 2016, 18(8): 1590-1603.
[46] LI Y, WANG R P, SHAN S G, et al. Hierarchical Hybrid Statistic Based Video Binary Code and Its Application to Face Retrieval in TV-Series // Proc of the IEEE International Conference and Workshops on Automatic Face and Gesture Recognition. Washington, USA: IEEE, 2015. DOI: 10.1109/FG.2015.7163089.
[47] LIONG V E, LU J W, TAN Y P, et al. Deep Video Hashing. IEEE Transactions on Multimedia, 2017, 19(6): 1209-1219.
[48] CHEN Z X, LU J W, FENG J J, et al. Nonlinear Structural Ha-shing for Scalable Video Search. IEEE Transactions on Circuits and Systems for Video Technology, 2017. DOI: 10.1109/TCSVT.2017.669095.
[49] WANG J, KUMAR S, CHANG S F. Semi-supervised Hashing for Scalable Image Retrieval // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2010: 3424-3431.
[50] WANG J, KUMAR S, CHANG S F. Semi-supervised Hashing for Large-Scale Search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(12): 2393-2406.
[51] LU J W, LIONG V E, ZHOU J. Simultaneous Local Binary Feature Learning and Encoding for Homogeneous and Heterogeneous Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. DOI: 10.1109/TPAMI.2017.2737538.
[52] DUAN Y Q, LU J W, FENG J J, et al. Learning Rotation-Invariant Local Binary Descriptor. IEEE Transactions on Image Proce-ssing, 2017, 26(8): 3636-3651.
[53] DUAN Y Q, LU J W, FENG J J, et al. Context-Aware Local Binary Feature Learning for Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. DOI: 10.1109/TPAMI.2017.2710183.
[54] LU J W, LIONG V E, ZHOU J. Cost-Sensitive Local Binary Feature Learning for Facial Age Estimation. IEEE Transactions on Image Processing, 2015, 24(12): 5356-5368.
[55] REN S Q, CAO X D, WEI Y C, et al. Face Alignment at 3000 FPS via Regressing Local Binary Features // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 1685-1692.
[56] GONG Y C, KUMAR S, ROWLEY H A, et al. Learning Binary Codes for High-Dimensional Data Using Bilinear Projections // Proc of the IEEE Conference on Computer Vision and Pattern Re-cognition. Washington, USA: IEEE, 2013: 484-491.
[57] BABENKO A, LEMPITSKY V. Additive Quantization for Extreme Vector Compression // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 931-938.
[58] XIA Y, HE K M, KOHLI P, et al. Sparse Projections for High-Dimensional Binary Codes // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 3332-3339.
[59] GONG Y C, LAZEBNIK S. Iterative Quantization: A Procrustean Approach to Learning Binary Codes // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2011: 817-824.
[60] YU F X, KUMAR S, GONG Y C, et al. Circulant Binary Embe-dding[C/OL]. [2017-11-25]. https://arxiv.org/pdf/1405.3162.pdf.
[61] SHEN F M, ZHOU X, YANG Y, et al. A Fast Optimization Me-thod for General Binary Code Learning. IEEE Transactions on Image Processing, 2016, 25(12): 5610-5621.
[62] LEUTENEGGER S, CHLI M, SIEGWART R Y. BRISK: Binary Robust Invariant Scalable Keypoints // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2011: 2548-2555.
[63] CALONDER M, LEPETIT V, STRECHA C, et al. Brief: Binary Robust Independent Elementary Features // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2010: 778-792.
[64] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: An Efficient Alternative to SIFT or SURF // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2011: 2564-2571.
[65] HUANG Z D, WEI Z Z, ZHANG G J. RWBD: Learning Robust Weighted Binary Descriptor for Image Matching. IEEE Transactions on Circuits and Systems for Video Technology, 2017. DOI: 10.1109/TCSVT.2017.2656471.
[66] BALNTAS V, TANG L L, MIKOLAJCZYK K. Bold-Binary Online Learned Descriptor for Efficient Image Matching // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 2367-2375.
[67] TRZCINSKI T, LEPETIT V. Efficient Discriminative Projections for Compact Binary Descriptors // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2012, I: 228-242.
[68] FAN B, KONG Q Q, TRZCINSKI T, et al. Receptive Fields Selection for Binary Feature Description. IEEE Transactions on Image Processing, 2014, 23(6): 2583-2595.
[69] BLAKE A, ISARD M. The CONDENSATION Algorithm-Conditional Density Propagation and Applications to Visual Tracking // Proc of the International Conference on Neural Information Proce-ssing Systems. Washington, USA: IEEE, 1996: 361-365.
[70] ROSS D A, LIM J, LIN R S, et al. Incremental Learning for Robust Visual Tracking. International Journal of Computer Vision, 2008, 77(1/2/3): 125-141.
[71] BABENKO B, YANG M H, BELONGIE S. Visual Tracking with Online Multiple Instance Learning // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2009: 983-990.
[72] LI X, SHEN C H, DICK A, et al. Learning Compact Binary Codes for Visual Tracking // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 2419-2426.
[73] JAIN H, ZEPEDA J, PREZ P, et al. SUBIC: A Supervised, Structured Binary Code for Image Search // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 833-842.