Abstract The deep convolutional neural network method can hardly analyze specific regions of an image and the relationship between those regions. A method for image aesthetic quality assessment is proposed by means of complementary combination of deep and handcrafted features. The specific regions dominating the aesthetic quality of the image are identified. Then, five groups of aesthetic relevant handcrafted features including line angles feature and clarity comparison feature are selected and designed. The deep features are acquired using Siamese network. Support vector regression algorithm is then applied to evaluate the score of the aesthetic quality of the image based on those handcrafted and deep features. The score is adjusted and finalized in light of the weight of Spearman rank-order correlation coefficient. Experimental results show that the proposed method outperforms the existing methods and the result is consistent with subjective assessment results.
Fund:Supported by National Natural Science Foundation of China(No.61502105)
About author:: (XIE Yanjuan, born in 1994, master stu-dent. Her research interests include image processing.) (CHEN Zhaojiong(Corresponding author), born in 1964, master, professor. Her research interests include image processing and computational intelligence.) (YE Dongyi, born in 1964, Ph.D., profe-ssor. His research interests include computational intelligence and data mining.)
[1] 王伟凝,蚁静缄,贺前华.可计算图像美学研究进展.中国图象图形学报, 2012, 17(8): 893-901. (WANG W N, YI J J, HE Q H. Review for Computational Image Aesthetics. Journal of Image and Graphics, 2012, 17(8): 893-901.) [2] LU X, LIN Z, SHEN X C, et al. Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 990-998. [3] DATTA R, JOSHI D, LI J, et al. Studying Aesthetics in Photographic Images Using a Computational Approach // Proc of the European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2006: 288-301. [4] KE Y, TANG X O, JING F, et al. The Design of High-Level Features for Photo Quality Assessment // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2006, I: 419-426. [5] LUO Y W, TANG X O. Photo and Video Quality Evaluation: Focusing on the Subject // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2008: 386-399. [6] WONG L K, LOW K L. Saliency-Enhanced Image Aesthetics Class Prediction // Proc of the 16th IEEE International Conference on Image Processing. Washington, USA: IEEE, 2009: 993-996. [7] TANG X O, LUO W, WANG X G. Content-Based Photo Quality Assessment. IEEE Transactions on Multimedia, 2013, 15(8): 1930-1943. [8] LU X, LIN Z, JIN H L, et al. Rating Image Aesthetics Using Deep Learning. IEEE Transactions on Multimedia, 2015, 17(11): 2021-2034. [9] DONG Z, TIAN X M. Multi-level Photo Quality Assessment with Multi-view Features. Neurocomputing, 2015, 168: 308-319. [10] 王伟凝,王 励,赵明权,等.基于并行深度卷积神经网络的图像美感分类.自动化学报, 2016, 42(6): 904-914. (WANG W N, WANG L, ZHAO M Q, et al. Image Aesthetic Classification Using Parallel Deep Convolutional Neural Networks. Acta Automatica Sinica, 2016, 42(6): 904-914.) [11] MAI L, JIN H L, LIU F. Composition-Preserving Deep Photo Aesthetics Assessment // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 497-506. [12] KONG S, SHEN X H, LIN Z, et al. Photo Aesthetics Ranking Network with Attributes and Content Adaptation // Proc of the European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2016: 662-679. [13] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet Cla-ssification with Deep Convolutional Neural Networks. Communications of the ACM, 2017, 60(6): 84-90. [14] SIMOND F, ARVANITOPOULOS N, S SSTRUNK S. Image Aesthetics Depends on Context // Proc of the IEEE International Conference on Image Processing. Washington, USA: IEEE, 2015: 3788-3792. [15] YANG C, ZHANG L H, LU H C, et al. Saliency Detection via Graph-Based Manifold Ranking // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 3166-3173. [16] 戚忠其.试论摄影作品中突出主体的技巧和方法.中国科技博览, 2012(2): 230. (QI Z Q. On the Skills and Methods of Stressing the Subject in Photographic Works. China Science and Technology Review, 2012(2): 230.) [17] 应凌楷,李子印,张聪聪.融合梯度信息与HVS滤波器的无参考清晰度评价.中国图象图形学报, 2015, 20(11): 1446-1452. (YING L K, LI Z Y, ZHANG C C. No-reference Sharpness Assessment with Fusion of Gradient Information and HVS Filter. Journal of Image and Graphics, 2015, 20(11): 1446-1452.) [18] MURRAY N, MARCHESOTTI L, PERRONNIN F. AVA: A Large-Scale Database for Aesthetic Visual Analysis // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2012: 2408-2415.
2017, Vol. 30 
