Global-Local Feature Extraction Method for Fine-Grained National Clothing Image Retrieval
ZHOU Qianqian1, LIU Li1,2, LIU Lijun1,2, FU Xiaodong1,2, HUANG Qingsong1,2
1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500; 2. Computer Technology Application Key Laboratory of Yunnan Province, Kunming University of Science and Technology, Kunming 650500
Abstract The low accuracy of fine-grained retrieval of national clothing images is caused by different clothing styles, accessories and patterns of national clothing. To address is problem, a global-local feature extraction method for fine-grained clothing image retrieval is proposed. Firstly, the input image is detected to obtain the foreground, styles, accessories and patterns images based on semantic labels of national clothing. Secondly, a multi-branch feature extraction model based on fully convolutional network is constructed to extract features from clothing images in different regions and obtain convolutional features of global, styles, accessories and patterns. Finally, the preliminary retrieval results are obtained by applying a similarity measure to the global features. Then,re-ranking of the result is performed by the local features of top 50 retrieval results and the query image. The final retrieval results are output by the result of re-ranking. The experimental results on the constructed national clothing image dataset show that the proposed method improves the accuracy of national clothing image retrieval effectively.
Fund:National Natural Science Foundation of China(No.61862036,61962030,81860318), Young Academic and Tech-nical Leader Candidate Program of Yunnan Province(No.201905C160046)
Corresponding Authors:
LIU Li, Ph.D., professor. Her research interests include computer-aided design and computer graphics, image processing and computer vision.
About author:: ZHOU Qianqian, master student. His research interests include computer vision and image processing.FU Xiaodong, Ph.D., professor. His research interests include services computing and decision theory and technology.LIU Lijun, Ph.D., associate professor. His research interests include image proce-ssing, cloud computing and information retrieval.HUANG Qingsong, master, professor. His research interests include machine learning, data mining and intelligent information system.
[1] WANG X,SUN Z F,ZHANG W Q,et al.Matching User Photos to Online Products with Robust Deep Features//Proc of the ACM International Conference on Multimedia Retrieval.New York,USA:ACM,2016:7-14. [2] 黄冬艳,刘 骊,付晓东,等.联合分割和特征匹配的服装图像检索.计算机辅助设计与图形学学报,2017,29(6):1075-1084. (HUANG D Y,LIU L,FU X D,et al.Clothing Retrieval via Co-segmentation and Feature Matching.Journal of Computer-Aided Design and Computer Graphics,2017,29(6):1075-1084.) [3] LI H J,WANG X H,TANG J H,et al. Combining Global and Local Matching of Multiple Features for Precise Item Image Retrieval.Multimedia Systems,2013,19(1):37-49. [4] KIAPOUR M H,HAN X F,LAZEBNIK S,et al.Where to Buy It:Matching Street Clothing Photos in Online Shops//Proc of the IEEE International Conference on Computer Vision.Washington,USA:IEEE,2015:3343-3351. [5] WANG X W,ZHANG T.Clothes Search in Consumer Photos via Color Matching and Attribute Learning//Proc of the 19th ACM International Conference on Multimedia.New York,USA:ACM,2011:1353-1356. [6] HUANG J S,FERIS R S,CHEN Q,et al.Cross-Domain Image Retrieval with a Dual Attribute-Aware Ranking Network//Proc of the IEEE International Conference on Computer Vision.Washington,USA:IEEE,2015:1062-1070. [7] LIU Z W,LUO P,QIU S,et al. DeepFashion:Powering Robust Clothes Recognition and Retrieval with Rich Annotations//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:1096-1104. [8] CHENG Z Q,WU X,LIU Y,et al.Video2Shop:Exact Matching Clothes in Videos to Online Shopping Images//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2017:4048-4056. [9] WANG Z H,GU Y J,ZHANG Y,et al.Clothing Retrieval with Visual Attention Model//Proc of the IEEE Conference on Visual Communications and Image Processing.Washington,USA:IEEE,2017.DOI:10.1109/VCIP.2017.8305144. [10] KUANG Z H,GAO Y M,LI G B,et al.Fashion Retrieval via Graph Reasoning Networks on a Similarity Pyramid//Proc of the IEEE/CVF International Conference on Computer Vision.Washing-ton,USA:IEEE,2019:3066-3075. [11] 王志伟,普园媛,王 鑫,等.基于多特征融合的多尺度服装图像精准化检索.计算机学报,2020,43(4):740-754. (WANG Z W,PU Y Y,WANG X,et al.Accurate Retrieval of Multi-scale Clothing Images Based on Multi-feature Fusion.Chinese Journal of Computers,2020,43(4):740-754.) [12] 林计文,刘华文,郑忠龙.面向图像检索的深度汉明嵌入哈希.模式识别与人工智能,2020,33(6):542-550. (LIN J W,LIU H W,ZHENG Z L.Deep Hamming Embedding Based Hashing for Image Retrieval.Pattern Recognition and Artificial Intelligence,2020,33(6):542-550.) [13] LUO Y D,WANG Z W,HUANG Z,et al.Snap and Find:Deep Discrete Cross-Domain Garment Image Retrieval[C/OL].[2020-08-22].https://arxiv.org/pdf/1904.02887.pdf. [14] HUO P,WANG Y H,LIU Q J.A Part-Based and Feature Fusion Method for Clothing Classification//Proc of the Pacific Rim Conference on Multimedia.Berlin,Germany:Springer,2016:231-241. [15] SUN Y H,LIU Q J.Attribute Recognition from Clothing Using a Faster R-CNN Based Multitask Network.International Journal of Wavelets,Multiresolution and Information Processing,2018,16(2).DOI:10.1142/S021969131840009X. [16] PEDRONETTE D C G E,TORRES R D S.Image Re-ranking and Rank Aggregation Based on Similarity of Ranked Lists//Proc of the International Conference on Computer Analysis of Images and Patterns.Berlin,Germany:Springer,2011:369-376. [17] YANG F,MATEI B,DAVIS L S.Re-ranking by Multi-feature Fusion with Diffusion for Image Retrieval//Proc of the IEEE Winter Conference on Applications of Computer Vision.Washington,USA:IEEE,2015:572-579. [18] YU T,WU Y W,BHATTACHARJEE S,et al. Efficient Object Instance Search Using Fuzzy Objects Matching//Proc of the 31st AAAI Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2017:4320-4326. [19] REN S Q,HE K M,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks.IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,39(6):1137-1149. [20] 钟茂兰,范 朴.中国少数民族服饰.北京:中国纺织出版社,2006. (ZHONG M L,FAN P,Chinese Minority Costumes.Beijing,China:China Textile Press,2006.) [21] RUSSELL B C,TORRALBA A,MURPHY K P,et al.LabelMe:A Database and Web-Based Tool for Image Annotation.International Journal of Computer Vision,2008,77(1/2/3):157-173. [22] 吴圣美,刘 骊,付晓东,等.结合人体检测和多任务学习的少数民族服装识别.中国图象图形学报,2019,24(4):562-572. (WU S M,LIU L,FU X D,et al.Human Detection and Multi-task Learning for Minority Clothing Recognition.Journal of Image and Graphics,2019,24(4):562-572.) [23] GE Y Y,ZHANG R M,WANG X G,et al.DeepFashion2:A Versatile Benchmark for Detection,Pose Estimation,Segmentation and Re-identification of Clothing Images//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2019:5337-5345.
2021, Vol. 34 
