Abstract The 3D object recognition is usually based on convolutional neural networks. The spatial information of 3D objects is lost due to too many pooling layers used in the process of feature aggregation. To solve the above problem, a convolutional vector network for 3D mesh object recognition is proposed in this paper. Firstly, the surface polynomials are introduced to fit the local mesh of the object, and then the surface shape convolution kernels are clustered. The feature vector of structure-aware is generated by self-measuring the similarity between the local mesh in the object and convolution kernels. The multi-headed attention mechanism module is then employed to achieve feature aggregation from local regions to a larger scale to obtain component-level feature vectors. Finally, the 3D objects are classified through the 3D vector network. The proposed network achieves a high classification accuracy on SHREC10, SHREC11 and SHREC15 datasets. In addition, generalization and robustness of the proposed network are demonstrated through the multi-resolution object comparison experiment and the multi-sampling point comparison experiment.
Fund:National Natural Science Foundation of China(No.62071260,62006131,61603202), Natural Science Foundation of Zhejiang Province(No.LZ22F020001,LY17F030002,LY20F030005)
Corresponding Authors:ZHAO Jieyu, Ph.D., professor. His research interests include computational intelligence, pattern re-cognition and natural human-computer interaction.
About author:: QIU Qilu, master student. His research interests include three-dimensional convolutio-nal networks and pattern recognition. CHEN Yu, Ph.D. candidate. Her research interests include image processing and deep learning.
[1] HANOCKA R, HERTZ A, FISH N, et al. MeshCNN: A Network with an Edge. ACM Transactions on Graphics, 2019, 38(4): 1-12. [2] SAUPE D, VRANIĆ D V.3D Model Retrieval with Spherical Harmonics and Moments // Proc of the Joint Pattern Recognition Symposium. Berlin,Germany: Springer, 2001: 392-397. [3] PAQUET E, RIOUX M.Nefertiti: A Tool for 3-D Shape Databases Management. SAE Transactions, 1999, 108: 387-393. [4] SHI B G, BAI S, ZHOU Z C, et al. DeepPano: Deep Panoramic Representation for 3-D Shape Recognition. IEEE Signal Processing Letters, 2015, 22(12): 2339-2343. [5] SINHA A, BAI J, RAMANI K.Deep Learning 3D Shape Surfaces Using Geometry Images // Proc of the European Conference on Computer Vision. Berlin,Germany: Springer, 2016: 223-240. [6] LENG B, ZHANG C, ZHOU X C, et al. Learning Discriminative 3D Shape Representations by View Discerning Networks. IEEE Transactions on Visualization and Computer Graphics, 2019, 25(10): 2896-2909. [7] WU Z R, SONG S R, KHOSLA A, et al. 3D ShapeNets: A Deep Representation for Volumetric Shapes // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington,USA: IEEE, 2015: 1912-1920. [8] BROCK A, LIM T, RITCHIE J M, et al. Generative and Discriminative Voxel Modeling with Convolutional Neural Networks[C/OL].[2021-10-21]. https://arxiv.org/pdf/1608.04236.pdf. [9] QI C R, SU H, MO K C, et al. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington,USA: IEEE, 2017: 77-85. [10] LAN S Y, YU R C, YU G, et al. Modeling Local Geometric Structure of 3D Point Clouds Using Geo-CNN // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington,USA: IEEE, 2019: 998-1008. [11] VAKALOPOULOU M, CHASSAGNON G, BUS N, et al. AtlasNet: Multi-atlas Non-linear Deep Networks for Medical Image Segmentation // Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin,Germany: Springer, 2018: 658-666. [12] KRIZHEVSKY A, SUTSKEVER I, HINTON G E.ImageNet Cla-ssification with Deep Convolutional Neural Networks. Communications of the ACM, 2017, 60(6): 84-90. [13] ZHANG X N, LUO P C, HU X W, et al. Research on Classification Performance of Small-Scale Dataset Based on Capsule Network // Proc of the 4th International Conference on Robotics and Artificial Intelligence. New York,USA: ACM, 2018: 24-28. [14] HINTON G E, KRIZHEVSKY A, WANG S D.Transforming Auto-Encoders // Proc of the International Conference on Artificial Neural Networks. Berlin,Germany: Springer, 2011: 44-51. [15] SABOUR S, FROSST N, HINTON G E.Dynamic Routing between Capsules ∥ Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2017: 3859-3869. [16] REN H, SU J L, LU H.Evaluating Generalization Ability of Con-volutional Neural Networks and Capsule Networks for Image Cla-ssification via Top-2 Classification[C/OL]. [2021-10-21].https://arxiv.org/pdf/1901.10112v1.pdf. [17] IESMANTAS T, ALZBUTAS R.Convolutional Capsule Network for Classification of Breast Cancer Histology Images // Proc of the International Conference on Image Analysis and Recognition. Berlin,Germany: Springer, 2018: 853-860. [18] JAISWAL A, ABDALMAGEED W, WU Y, et al. CapsuleGAN: Generative Adversarial Capsule Network // Proc of the European Conference on Computer Vision. Berlin,Germany: Springer, 2018: 526-535. [19] ZHAO W, YE J B, YANG M, et al. Investigating Capsule Networks with Dynamic Routing for Text Classification // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg,USA: ACL, 2018: 3110-3119. [20] NGUYEN H H, YAMAGISHI J, ECHIZEN I.Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos // Proc of the IEEE International Conference on Acoustics, Speech and Signal Processing. Washington,USA: IEEE, 2019: 2307-2311. [21] CHEN Y, ZHAO J Y, SHI C W, et al. Mesh Convolution: A Novel Feature Extraction Method for 3D Nonrigid Object Classification. IEEE Transactions on Multimedia, 2020, 23: 3098-3111. [22] PATRICK M K, ADEKOYA A F, MIGHTY A A, et al. Capsule Networks-A Survey. Journal of King Saud University-Computer and Information Sciences, 2022, 34(1): 1295-1310. [23] VASWANI A, SHAZEER N, PARMAR N, et al.Attention Is All You Need // Proc of the 31st Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2017: 6000-6010. [24] SUN G C, DING S F, SUN T F, et al. SA-CapsGAN: Using Ca-psule Networks with Embedded Self-Attention for Generative Adversarial Network. Neurocomputing, 2021, 423: 399-406. [25] CARION N, MASSA F, SYNNAEVE G, et al. End-to-End Object Detection with Transformers // Proc of the European Conference on Computer Vision. Berlin,Germany: Springer, 2020: 213-229. [26] BU S H, LIU Z B, HAN J W, et al. Learning High-Level Feature by Deep Belief Networks for 3-D Model Retrieval and Recognition. IEEE Transactions on Multimedia, 2014, 16(8): 2154-2167. [27] GHODRATI H, HAMZA A B.Nonrigid 3D Shape Retrieval Using Deep Auto-Encoders. Applied Intelligence, 2017, 47(1): 44-61. [28] MASOUMI M, LI C Y, HAMZA A B.A Spectral Graph Wavelet Approach for Nonrigid 3D Shape Retrieval. Pattern Recognition Letters, 2016, 83: 339-348. [29] XIE J, DAI G X, ZHU F, et al. DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(7): 1335-1345. [30] LUCIANO L, HAMZA A B.Deep Learning with Geodesic Moments for 3D Shape Classification. Pattern Recognition Letters, 2018, 105: 182-190. [31] 韩丽,朴京钰,兰鹏燕,等.结构感知深度学习的三维形状分类方法.计算机辅助设计与图形学学报, 2021, 33(1): 29-38. (HAN L, PIAO J Y, LAN P Y, et al. 3D Shape Classification Me-thod Based on Shape-Aware Deep Learning. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(1): 29-38.) [32] SU H, MAJI S, KALOGERAKIS E, et al. Multi-view Convolutional Neural Networks for 3D Shape Recognition // Proc of the IEEE International Conference on Computer Vision. Washington,USA: IEEE, 2015: 945-953. [33] QI C R, YI L, SU H, et al.PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space // Proc of the 31st Conference on Neural Information Processing Systems. Cambridge, USA: The MIT Press, 2017: 2105-5114. [34] XU Y F, FAN T Q, XU M Y, et al. SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters // Proc of the European Conference on Computer Vision. Berlin,Germany: Springer, 2018: 90-105. [35] VERMA N, BOYER E, VERBEEK J.FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington,USA: IEEE, 2018: 2598-2606. [36] YANG Y Q, LIU S L, PAN H, et al. PFCNN: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington,USA: IEEE, 2020: 13575-13584. [37] TAN C Q, SUN F C, KONG T, et al. A Survey on Deep Transfer Learning // Proc of the International Conference on Artificial Neural Networks. Berlin,Germany: Springer, 2018: 270-279.
2022, Vol. 35 
