Self-Supervised Non-Isometric 3D Shape Collection Correspondence Calculation Method
WU Yan1,2, YANG Jun1,3, ZHANG Siyang1
1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070; 2. School of Big Data and Artificial Intelligence, Fujian Polytechnic Normal University, Fuqing 350300; 3. Faculty of Geomatics, Lanzhou Jiaotong University, Lanzhou 730070
Abstract Aiming at the problem of low accuracy and poor generalization ability in existing non-isometric 3D shape collection correspondence calculation methods, a self-supervised non-isometric 3D shape collection correspondence calculation method using deep intrinsic-extrinsic feature alignment algorithm is proposed. Firstly, discriminative feature descriptors are obtained by directly learning the original 3D shape features through DiffusionNet. Then, the deep intrinsic-extrinsic feature alignment algorithm is employed to compute correspondences between non-isometric shapes. Consistency between internal and external information is realized by utilizing local manifold harmonic bases as intrinsic information of the shapes and integrating external information such as Cartesian coordinates. Consequently, correspondence results are generated automatically in an unsupervised manner. Finally, a weighted undirected graph of non-isometric shape collections is constructed. Based on the principle of inherent correlation among similar geometric shapes, a self-supervised multi-shape matching algorithm is designed to continuously enhance the cycle-consistency of the shortest path in the shape graph, and thus optimal correspondences for non-isometric 3D shape collections are obtained. Experimental results demonstrate that the proposed method achieves small geodesic errors in correspondences with accurate results, and effectively deals with the symmetric ambiguity problem with good generalization ability.
Fund:National Natural Science Foundation of China(No.61773301), Natural Science Foundation of Fujian Province(No.2022J01972), 2021 Central Government Funds for Guiding Local Science and Technology Development(No.2021-51)
Corresponding Authors:
YANG Jun, Ph.D., professor. His research interests include computer graphics, remote sensing image analysis and processing and deep learning.
About author:: WU Yan Ph.D. candidate. His research interests include 3D shape correspondence and deep learning.ZHANG Siyang, Master. His research interests include machine learning and software defect prediction.
[1] VAN KAICK O, ZHANG H, HAMARNEH G, et al. A Survey on Shape Correspondence. Computer Graphics Forum, 2011, 30(6): 1681-1707. [2] SAHILLIOĞLU Y. Recent Advances in Shape Correspondence. The Visual Computer, 2020, 36(8): 1705-1721. [3] DENG B L, YAO Y X, DYKE R M, et al. A Survey of Non-Rigid 3D Registration. Computer Graphics Forum, 2022, 41(2): 559-589. [4] COSMO L, RODOLÀ E, MASCI J, et al. Matching Deformable Objects in Clutter // Proc of the 4th International Conference on 3d Vi-sion. Washington, USA: IEEE, 2016. DOI: 10.1109/3DV.2016.10. [5] OVSJANIKOV M, BEN-CHEN M, SOLOMON J, et al. Functional Maps: A Flexible Representation of Maps Between Shapes. ACM Transactions on Graphics, 2012, 31(4). DOI: 10.1145/2185520.2185526. [6] MELZI S, RODOLÀ E, CASTELLANI U, et al. Localized Manifold Harmonics for Spectral Shape Analysis. Computer Graphics Forum, 2018, 37(6): 20-34. [7] MANDAD M, COHEN-STEINER D, KOBBELT L, et al. Variance-Minimizing Transport Plans for Inter-Surface Mapping. ACM Tran-sactions on Graphics, 2017, 36(4). DOI: 10.1145/3072959.3073671. [8] EZUZ D, SOLOMON J, BEN-CHEN M.Reversible Harmonic Maps between Discrete Surfaces. ACM Transactions on Graphics, 2019, 38(2). DOI: 10.1145/3202660. [9] EDELSTEIN M, EZUZ D, BEN-CHEN M.ENIGMA: Evolutionary Non-Isometric Geometry Matching. ACM Transactions on Graphics,2020, 39(4). DOI: 10.1145/3386569.3392447. [10] EISENBERGER M, LAHNER Z, CREMERS D.Smooth Shells: Multi-scale Shape Registration with Functional Maps // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 12262-12271. [11] PANINE M, KIRGO M, OVSJANIKOV M.Non-Isometric Shape Matching via Functional Maps on Landmark-Adapted Bases. Computer Graphics Forum, 2022, 41(6): 394-417. [12] SAHILLIOĞLU Y, YEMEZ Y. Multiple Shape Correspondence by Dynamic Programming. Computer Graphics Forum, 2014, 33(7): 121-130. [13] 杨军,雷鸣.结合函数映射与循环一致性约束的模型簇对应关系计算.激光与光电子学进展, 2019, 56(8): 123-133. (YANG J, LEI M.Correspondence Calculation of Model Cluster by Functional Mapping combined with Cycle-Consistency Constraints. Laser and Optoelectronics Progress, 2019, 56(8): 123-133.) [14] HUANG R Q, REN J, WONKA P, et al. CONSISTENT ZOOM-OUT: Efficient Spectral Map Synchronization. Computer Graphics Forum, 2020, 39(5): 265-278. [15] MELZI S, REN J, RODOLÀ E, et al. ZOOMOUT: Spectral Upsampling for Efficient Shape Correspondence. ACM Transactions on Graphics, 2019, 38(6). DOI: 10.1145/3355089.3356524. [16] HUANG R Q, ACHLIOPTAS P, GUIBAS L, et al. Limit Shapes-A Tool for Understanding Shape Differences and Variability in 3D Model Collections. Computer Graphics Forum, 2019, 38(5): 187-202. [17] GAO M L, LAHNER Z, THUNBERG J, et al. Isometric Multi-Shape Matching // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 14178-14188. [18] 杨军,李金泰,高志明.无监督的三维模型簇对应关系协同计算.浙江大学学报(工学版), 2022, 56(10): 1935-1947. (YANG J, LI J T, GAO Z M.Unsupervised Co-calculation on Correspondence of Three-Dimensional Shape Collections. Journal of Zhejiang University(Engineering Science), 2022, 56(10): 1935-1947.) [19] CAO D L, BERNARD F.Unsupervised Deep Multi-Shape Mat-ching // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 55-71. [20] LITANY O, RODOLÀ E, BRONSTEIN A M, et al. Non-Rigid Puzzles. Computer Graphics Forum, 2016, 35(5): 135-143. [21] WU Y, YANG J.Multi-part Shape Matching by Simultaneous Par-tial Functional Correspondence. The Visual Computer, 2023, 39(6): 393-412. [22] SHARP N, ATTAIKI S, CRANE K,et al. DiffusionNet: Discretization Agnostic Learning on Surfaces. ACM Transactions on Gra-phics, 2022, 41(3). DOI: 10.1145/3507905. [23] MYRONENKO A, SONG X B.Point Set Registration: Coherent Point Drift. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(12): 2262-2275. [24] 段凡丁. 关于最短路径的 SPFA 快速算法.西南交通大学学报, 1994, 29(2): 207-212. (DUAN F D.A Faster Algorithm for Shortest-Path-SPFA. Journal of Southwest Jiaotong University, 1994, 29(2): 207-212.) [25] 吴衍,杨军.内外特征一致性对齐的非等距三维模型对应关系计算.计算机辅助设计与图形学学报, 2023, 35(5): 749-759. (WU Y, YANG J.Correspondence Calculation of Non-Isometric 3D Shapes by Intrinsic-Extrinsic Feature Alignment. Journal of Computer-Aided Design and Computer Graphics, 2023, 35(5): 749-759.) [26] SORKINE O, ALEXA M.As-Rigid-As-Possible Surface Modeling[C/OL]. [2023-11-09].https://www.cs.jhu.edu/~misha/ReadingSeminar/Papers/Sorkine07.pdf. [27] KOVNATSKY A, GLASHOFF K, BRONSTEIN M M.MADMM: A Generic Algorithm for Non-Smooth Optimization on Manifolds // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 680-696. [28] SUN M Z, MAO S W, JIANG P H, et al. Spatially and Spectrally Consistent Deep Functional Maps // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2023: 14451-14561. [29] MAGNET R, OVSJANIKOV M.Scalable and Efficient Functional Map Computations on Dense Meshes. Computer Graphics Forum, 2023, 42(2): 89-101. [30] ZUFFI S, KANAZAWA A, JACOBS D W, et al. 3D Menagerie: Modeling the 3D Shape and Pose of Animals // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 5524-5532. [31] MAGNET R, REN J, SORKINE-HORNUNG O, et al. Smooth Non-Rigid Shape Matching via Effective Dirichlet Energy Optimization // Proc of the International Conference on 3D Vision. Washington, USA: IEEE, 2022: 495-504. [32] LÄHNER Z, RODOLÀ E, BRONSTEIN M M, et al. SHRCE'16: Matching of Deformable Shapes with Topological Noise // Proc of the Eurographics Workshop on 3D Object Retrieval. Berlin, Germany: Springer, 2016: 55-60. [33] MELZI S, MARIN R, RODOLÀ E, et al. SHREC 2019: Mat-ching Humans with Different Connectivity // Proc of the Eurographics Workshop on 3D Object Retrieval. Berlin, Germany: Springer, 2019: 121-128. [34] BOGO F, ROMERO J, LOPER M, et al. FAUST: Dataset and Eva-luation for 3D Mesh Registration // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 3794-3801. [35] ANGUELOV D, SRINIVASAN P, KOLLER D, et al. SCAPE: Shape Completion and Animation of People. ACM Transactions on Graphics, 2005, 24(3): 408-416. [36] LI Y, TAKEHARA H, TAKETOMI T, et al. 4DComplete: Non-Rigid Motion Estimation beyond the Observable Surface // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2021: 12686-12696. [37] KIM V G, LIPMAN Y, FUNKHOUSER T.3 Blended Intrinsic Maps. ACM Transactions on Graphics, 2011, 30(4). DOI: 10.1145/2010324.1964974. [38] REN J, POULENARD A, WONKA P, et al. Continuous and Orien-tation-Preserving Correspondences via Functional Maps. ACM Transactions on Graphics, 2018, 37(6). DOI: 10.1145/3272127.3275040.
2024, Vol. 37 
