Abstract For the image retrieval system based on spatial relationship of objects in images,it is hard to automatically recognize objects and their spatial relations correctly. Based on the outputs of object detection algorithms,a triple representation of the spatial relationship in images is proposed. Based on the representation,a method for indexing images,computing similarities and ranking results is proposed. A 2D user-match interface is also developed for users to express their needs in terms of retrieval keywords and spatial relationships,and a prototype is established. The representation is robust against errors of object detection. Incorporating the confidence given by object detection into the triple representation and ranking method,the impact of object detection errors on the performance of image retrieval is reduced. With the queries comprising explicit spatial relationship,the proposed approach gives more accurate results in experiments. It performs better than the existing systems in terms of NDCG@m,MAP and F@m.
[1\]Chen C Y,Chang C C. An Object Oriented Similarity Retrieval Algorithm for Iconic Image Databases. Pattern Recognition Letters,1993,14(6): 465-470 [2]Chua T S,Lim S K,Pung H K. Content Based Retrieval of Segmented Images // Proc of the 2nd ACM International Conference on Multimedia. San Francisco,USA,1994: 211-218 [3]Egenhofer M J,Al Taha K K. Reasoning about Gradual Changes of Topological Relationships // Proc of the International Conference GIS-From Space to Territory: Theories and Methods of Spatio Temporal Reasoning. Pisa,Italy,1992: 196-219 [4]Egenhofer M J,Franzosa R. Point Set Topological Spatial Relations. International Journal of Geographical Information Systems,1991,5(2): 161-174 [5]Huang P W,Hsu L,Su Y W,et al. Spatial Inference and Similarity Retrieval of an Intelligent Image Database System Based on Object's Spanning Representation. Journal of Visual Languages and Computing,2008,19(6): 637-651 [6]Lee S Y,Hsu F J. Spatial Reasoning and Similarity Retrieval of Images Using 2D C String Knowledge Representation. Pattern Recognition,1992,25(3): 305-318 [7]Lee S Y,Shan M K,Yang W P. Similarity Retrieval of Iconic Image Database. Pattern Recognition,1989,22(6): 675-682 [8]Nabil M,Ngu A H H,Shepherd J. Picture Similarity Retrieval Using the 2D Projection Interval Representation. IEEE Trans on Knowledge and Data Engineering,1996,8(4): 533-539 [9]Zhou X M,Ang C H,Ling T W. Indexing for Multipoint Interactive Similarity Retrieval in Iconic Spatial Image Databases. Journal of Visual Languages and Computing,2008,19(1): 24-38 [10]Wang Yanlong,Liu Jinhua,Wang Liping. Research of Image Retrieval Method Based on Object Spatial Relationships.Computer Technology and Development,2006,16(1): 66-68 (in Chinese) (汪彦龙,刘金华,王丽萍.基于对象空间关系的图像检索方法研究.计算机技术与发展,2006,16(1): 66-68) [11]Huang P W,Jean Y R. Using 2D C+ Strings as Spatial Knowledge Representation for Image Database Systems. Pattern Recognition,1994,27(9): 1249-1257 [12]Chang Shikuo,Shi Qingyun,Yan Chengwen. Iconic Indexing by 2 D Strings. IEEE Trans on Pattern Analysis and Machine Intelligence,1987,9(3): 413-428 [13]Huang P W,Lee C H. Image Database Design Based on 9D SPA Representation for Spatial Relations. IEEE Trans on Knowledge and Data Engineering,2004,16(12): 1486-1496. [14]Xu Hao,Wang Jingdong,Hua Xiansheng,et al. Image Search by Concept Map // Proc of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Geneva,Switzerland,2010:275-282 [15]Felzenszwalb P F,Girshick R B,McAllester D,et al. Object Detection with Discriminatively Trained Part Based Models. IEEE Trans on Pattern Analysis and Machine Intelligence,2010,32(9): 1627-1645
2013, Vol. 26 
