AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We presented an extension to the functional maps framework by replacing the standard Laplace-Beltrami eigenfunctions with learned functions

Correspondence learning via linearly-invariant embedding

NIPS 2020, (2020)

Cited by: 0|Views36
EI
Full Text
Bibtex
Weibo

Abstract

In this paper, we propose a fully differentiable pipeline for estimating accurate dense correspondences between 3D point clouds. The proposed pipeline is an extension and a generalization of the functional maps framework. However, instead of using the Laplace-Beltrami eigenfunctions as done in virtually all previous works in this domain...More

Code:

Data:

0
Introduction
  • Computing correspondences between geometric objects is a widely investigated task. Its applications are countless: rigid and non-rigid registration methods are instrumental in engineering, medicine and biology [25, 29, 16] among other fields.
  • The non-rigid correspondence problem is challenging as a successful solution must deal with a large variability in shape deformations and be robust to noise in the input data
  • To address this problem, in recent years, several data-driven approaches have been proposed to learn the optimal transformation model from data rather than imposing it a priori, including [20, 61, 7] among others.
  • These methods have shown that optimal feature or descriptor functions can be learned from data and used successfully within the functional map pipeline to obtain accurate dense correspondences
Highlights
  • Computing correspondences between geometric objects is a widely investigated task
  • Point cloud registration is important for range scan data, e.g., in robotics [19, 52], but the problem can be generalized to abstract domains like graphs [58, 15]
  • A prominent direction is based on the functional map representation [39], which has been adapted to the learning-based setting [31, 22, 48, 13]
  • Inspired by the success and robustness of these techniques, we propose the first fully-differentiable functional maps pipeline, in which both the probe functions and the functional basis are learned from the data
  • Note that we describe our learning-based pipeline adapted to point clouds
  • We presented an extension to the functional maps framework by replacing the standard Laplace-Beltrami eigenfunctions with learned functions
Methods
  • The authors evaluate the pipeline on the correspondence problem between non-rigid 3D point clouds in the challenging class of human models
  • The authors use this class because of the availability of data and baselines for comparison but stress that the method is general and can be applied to any shape category.
  • For the experiments the authors train over 10K shapes from the SURREAL dataset [57], resampled at 1K vertices.
  • The authors report in Supplementary Materials the complete description of the architectures and the training data
Results
  • The authors demonstrate that the approach achieves state-of-the-art results in challenging non-rigid 3D point cloud correspondence applications.
Conclusion
  • While the basis and probe function networks appear similar as they both output a matrix, they are different in their losses and, as consequence, in the task that they solve.
  • By first obtaining a smooth embedding and using a small number of salient feature descriptors the approach allows to find a dense correspondence even in challenging cases, in which individual points may not be easy to distinguish.In this paper, the authors presented an extension to the functional maps framework by replacing the standard Laplace-Beltrami eigenfunctions with learned functions
  • The authors achieve this by learning an optimal linearly-invariant embedding and a separate network that aligns embeddings of different shapes.
  • The authors believe that these results only scratch the surface and can pave the way to future work on invariant embeddings for shape correspondence and other related problems
Related work
  • In addition to approaches mentioned above, here we briefly discuss previous works in the shape correspondence domain that are either closest to ours or most relevant for comparison and evaluation. We refer to the available surveys [5, 50] for a more complete overview.

    Functional maps The core of our method is the functional maps framework originally proposed in [39] which formulates the correspondence problem in the functional domain instead of the classical matching between points. In the functional space, a correspondence can be represented by a small matrix encoded in a reduced basis and computed as the optimal transformation that aligns a given set of probe functions possibly with other regularization. This method inspired a large number of further extensions, including [38, 14, 47, 46] to name a few. A more general overview of this area can be found in [40]. In our paper we also exploit the link between the functional representation and the adjoint map that has been originally developed in [24].
Funding
  • Parts of this work were supported by the KAUST OSR Award No CRG-2017-3426, the ERC Starting Grant No 758800 (EXPROTEA) and the ANR AI Chair AIGRETTE
Study subjects and analysis
pairs: 100
[47], which extends the functional maps framework to partial cases. In the middle we visualize a qualitative comparison on one of the 100 pairs tested, where the correspondence in encoded by the color transfer. We highlight that it is not always possible to have a transformation that produces a perfect matching

pairs: 100
The evaluation of the correspondence for point clouds generated from the FAUST dataset without or with additional noise. On the left, cumulative curves with mean error in the legends. On the right, a qualitative example in Noise setup, with the related hotmap error. Qantitative results on 100 pairs of the test set with 30% outlier points, compared to the baselines, with a qualitative example. A qualitative example of full shapes from FAUST and a fragmented version that matching between two statues. Despite consists of several small disconnected components. This the presence of clutter, partiality and nonexperiment tests how each basis is affected by heavy loss isometry our point cloud-based approach of geometry. Fixing a basis, we evaluate 1) the matching shows resilience. using a ground-truth transformation to retrieve the optimal linear transformation, on the left of Figure 5; 2) the correspondence estimated with the best pipeline for the given basis, on the right. The average geodesic errors are reported in the legends. In 2) for LBO we consider partial functional maps (PFM)

Reference
  • Scan the world project. https://www.myminifactory.com/scantheworld, [Online; accessed 22-October-2020]
    Findings
  • Atzmon, M., Maron, H., Lipman, Y.: Point convolutional neural networks by extension operators. ACM Trans. Graph. 37(4) (Jul 2018)
    Google ScholarLocate open access versionFindings
  • Aubry, M., Schlickewei, U., Cremers, D.: The wave kernel signature: A quantum mechanical approach to shape analysis. In: Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on. pp. 1626–163IEEE (2011)
    Google ScholarLocate open access versionFindings
  • Belkin, M., Sun, J., Wang, Y.: Constructing laplace operator from point clouds in rd. In: Proceedings of the twentieth annual ACM-SIAM symposium on Discrete algorithms. pp. 1031– 1040. Society for Industrial and Applied Mathematics (2009)
    Google ScholarLocate open access versionFindings
  • Biasotti, S., Cerri, A., Bronstein, A., Bronstein, M.: Recent trends, applications, and perspectives in 3d shape similarity assessment. Computer Graphics Forum 35(6), 87–119 (2016)
    Google ScholarLocate open access versionFindings
  • Bogo, F., Romero, J., Loper, M., Black, M.J.: FAUST: Dataset and evaluation for 3D mesh registration. In: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, NJ, USA (Jun 2014)
    Google ScholarLocate open access versionFindings
  • Boscaini, D., Masci, J., Rodolà, E., Bronstein, M.M., Cremers, D.: Anisotropic diffusion descriptors. Computer Graphics Forum 35(2), 431–441 (2016)
    Google ScholarLocate open access versionFindings
  • Choy, C.B., Gwak, J., Savarese, S., Chandraker, M.: Universal correspondence network. In: Advances in Neural Information Processing Systems. pp. 2414–2422 (2016)
    Google ScholarLocate open access versionFindings
  • Clarenz, U., Rumpf, M., Telea, A.: Finite elements on point based surfaces. In: Proceedings of the First Eurographics conference on Point-Based Graphics. pp. 201–211. SPBG’04, Eurographics Association, Goslar, DEU (2004)
    Google ScholarLocate open access versionFindings
  • Corman, É., Ovsjanikov, M., Chambolle, A.: Supervised descriptor learning for non-rigid shape matching. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) Computer Vision - ECCV 2014 Workshops. pp. 283–298. Springer International Publishing, Cham (2015)
    Google ScholarLocate open access versionFindings
  • Denitto, M., Melzi, S., Bicego, M., Castellani, U., Farinelli, A., Figueiredo, M.A.T., Kleiman, Y., Ovsjanikov, M.: Region-based correspondence between 3d shapes via spatially smooth biclustering. In: The IEEE International Conference on Computer Vision (ICCV) (Oct 2017)
    Google ScholarLocate open access versionFindings
  • Deprelle, T., Groueix, T., Fisher, M., Kim, V., Russell, B., Aubry, M.: Learning elementary structures for 3d shape generation and matching. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32, pp. 7435–7445. Curran Associates, Inc. (2019)
    Google ScholarLocate open access versionFindings
  • Donati, N., Sharma, A., Ovsjanikov, M.: Deep geometric functional maps: Robust feature learning for shape correspondence. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)
    Google ScholarLocate open access versionFindings
  • Ezuz, D., Ben-Chen, M.: Deblurring and denoising of maps between shapes. Computer Graphics Forum 36(5), 165–174 (2017)
    Google ScholarLocate open access versionFindings
  • Fey, M., Lenssen, J.E., Morris, C., Masci, J., Kriege, N.M.: Deep graph matching consensus. arXiv preprint arXiv:2001.09621 (2020)
    Findings
  • Gainza, P., Sverrisson, F., Monti, F., Rodola, E., Boscaini, D., Bronstein, M., Correia, B.: Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning. Nature Methods 17(2), 184–192 (2020)
    Google ScholarLocate open access versionFindings
  • Ganapathi-Subramanian, V., Thibert, B., Ovsjanikov, M., Guibas, L.: Stable region correspondences between non-isometric shapes. In: Proceedings of the Symposium on Geometry Processing. p. 121–133. SGP ’16, Eurographics Association, Goslar, DEU (2016)
    Google ScholarLocate open access versionFindings
  • Ginzburg, D., Raviv, D.: Cyclic functional mapping: Self-supervised correspondence between non-isometric deformable shapes. arXiv preprint arXiv:1912.01249 (2019)
    Findings
  • Gojcic, Z., Zhou, C., Wegner, J.D., Wieser, A.: The perfect match: 3d point cloud matching with smoothed densities. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)
    Google ScholarLocate open access versionFindings
  • Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: 3d-coded: 3d correspondences by deep deformation. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 230–246 (2018)
    Google ScholarLocate open access versionFindings
  • Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: Unsupervised cycle-consistent deformation for shape matching. In: Computer Graphics Forum. vol. 38, pp. 123–133. Wiley Online Library (2019)
    Google ScholarLocate open access versionFindings
  • Halimi, O., Litany, O., Rodola, E., Bronstein, A.M., Kimmel, R.: Unsupervised learning of dense shape correspondence. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)
    Google ScholarLocate open access versionFindings
  • Huang, Q., Wang, F., Guibas, L.: Functional map networks for analyzing and exploring large shape collections. ACM Transactions on Graphics (TOG) 33(4), 1–11 (2014)
    Google ScholarLocate open access versionFindings
  • Huang, R., Ovsjanikov, M.: Adjoint map representation for shape analysis and matching. In: Computer Graphics Forum. vol. 36, pp. 151–163. Wiley Online Library (2017)
    Google ScholarLocate open access versionFindings
  • Jin, Y.H., Lee, W.H.: Fast cylinder shape matching using random sample consensus in large scale point cloud. Applied Sciences 9(5), 974 (2019)
    Google ScholarLocate open access versionFindings
  • Kovnatsky, A., Bronstein, M.M., Bronstein, A.M., Glashoff, K., Kimmel, R.: Coupled quasiharmonic bases. In: Computer Graphics Forum. vol. 32, pp. 439–448. Wiley Online Library (2013)
    Google ScholarLocate open access versionFindings
  • Levy, B.: Laplace-beltrami eigenfunctions towards an algorithm that" understands" geometry. In: IEEE International Conference on Shape Modeling and Applications 2006 (SMI’06). pp. 13–13. IEEE (2006)
    Google ScholarLocate open access versionFindings
  • Levy, B., Zhang, R.H.: Spectral geometry processing. In: ACM SIGGRAPH Course Notes (2010)
    Google ScholarLocate open access versionFindings
  • Li, K., Chong, M.J., Liu, J., Forsyth, D.: Toward accurate and realistic virtual try-on through shape matching and multiple warps. arXiv preprint arXiv:2003.10817 (2020)
    Findings
  • Liang, J., Zhao, H.: Solving partial differential equations on point clouds. SIAM Journal on Scientific Computing 35(3), A1461–A1486 (2013)
    Google ScholarLocate open access versionFindings
  • Litany, O., Remez, T., Rodolà, E., Bronstein, A., Bronstein, M.: Deep functional maps: Structured prediction for dense shape correspondence. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 5659–5667 (2017)
    Google ScholarLocate open access versionFindings
  • Marin, R., Melzi, S., Rodolà, E., Castellani, U.: Farm: Functional automatic registration method for 3d human bodies. Computer Graphics Forum 39(1), 160–173 (2020)
    Google ScholarLocate open access versionFindings
  • Masci, J., Boscaini, D., Bronstein, M.M., Vandergheynst, P.: Geodesic convolutional neural networks on riemannian manifolds. In: Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCV). p. 832–840. ICCV’15, IEEE Computer Society, USA (2015)
    Google ScholarLocate open access versionFindings
  • Melzi, S., Marin, R., Musoni, P., Bardon, F., Tarini, M., Castellani, U.: Intrinsic/extrinsic embedding for functional remeshing of 3d shapes. Computers & Graphics 88, 1 – 12 (2020)
    Google ScholarLocate open access versionFindings
  • Melzi, S., Ren, J., Rodolà, E., Sharma, A., Wonka, P., Ovsjanikov, M.: Zoomout: Spectral upsampling for efficient shape correspondence. ACM Transactions on Graphics (TOG) 38(6), 155 (2019)
    Google ScholarLocate open access versionFindings
  • Melzi, S., Rodolà, E., Castellani, U., Bronstein, M.: Localized manifold harmonics for spectral shape analysis. Computer Graphics Forum 37(6), 20–34 (2018)
    Google ScholarLocate open access versionFindings
  • Nogneng, D., Melzi, S., Rodolà, E., Castellani, U., Bronstein, M., Ovsjanikov, M.: Improved functional mappings via product preservation. Computer Graphics Forum 37(2), 179–190 (2018)
    Google ScholarLocate open access versionFindings
  • Nogneng, D., Ovsjanikov, M.: Informative descriptor preservation via commutativity for shape matching. In: Computer Graphics Forum. vol. 36, pp. 259–267. Wiley Online Library (2017)
    Google ScholarLocate open access versionFindings
  • Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. ACM Transactions on Graphics (TOG) 31(4), 30:1–30:11 (2012)
    Google ScholarLocate open access versionFindings
  • Ovsjanikov, M., Corman, E., Bronstein, M., Rodolà, E., Ben-Chen, M., Guibas, L., Chazal, F., Bronstein, A.: Computing and processing correspondences with functional maps. In: SIGGRAPH 2017 Courses (2017)
    Google ScholarFindings
  • Pinkall, U., Polthier, K.: Computing discrete minimal surfaces and their conjugates. Experimental mathematics 2(1), 15–36 (1993)
    Google ScholarLocate open access versionFindings
  • Pokrass, J., Bronstein, A.M., Bronstein, M.M., Sprechmann, P., Sapiro, G.: Sparse modeling of intrinsic correspondences. In: Computer Graphics Forum. vol. 32, pp. 459–468. Wiley Online Library (2013)
    Google ScholarLocate open access versionFindings
  • Poulenard, A., Ovsjanikov, M.: Multi-directional geodesic neural networks via equivariant convolution. ACM Transactions on Graphics (TOG) 37(6), 1–14 (2018)
    Google ScholarLocate open access versionFindings
  • Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 652–660 (2017)
    Google ScholarLocate open access versionFindings
  • Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. p. 5105–5114. NIPS’17, Curran Associates Inc., Red Hook, NY, USA (2017)
    Google ScholarLocate open access versionFindings
  • Ren, J., Poulenard, A., Wonka, P., Ovsjanikov, M.: Continuous and orientation-preserving correspondences via functional maps. ACM Transactions on Graphics (TOG) 37(6), 1–16 (2018)
    Google ScholarLocate open access versionFindings
  • Rodolà, E., Cosmo, L., Bronstein, M.M., Torsello, A., Cremers, D.: Partial functional correspondence. In: Computer Graphics Forum. vol. 36, pp. 222–236. Wiley Online Library (2017)
    Google ScholarLocate open access versionFindings
  • Roufosse, J.M., Sharma, A., Ovsjanikov, M.: Unsupervised deep learning for structured shape matching. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1617–1627 (2019)
    Google ScholarLocate open access versionFindings
  • Rustamov, R.M.: Laplace-beltrami eigenfunctions for deformation invariant shape representation. In: Proceedings of the fifth Eurographics symposium on Geometry processing. pp. 225–233. Eurographics Association (2007)
    Google ScholarLocate open access versionFindings
  • Sahillioglu, Y.: Recent advances in shape correspondence. The Visual Computer pp. 1–17 (09 2019)
    Google ScholarFindings
  • Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. Computer Graphics Forum 28(5), 1383–1392 (2009)
    Google ScholarLocate open access versionFindings
  • Sánchez-Belenguer, C., Ceriani, S., Taddei, P., Wolfart, E., Sequeira, V.: Global matching of point clouds for scan registration and loop detection. Robotics and Autonomous Systems 123, 103324 (2020)
    Google ScholarLocate open access versionFindings
  • Taubin, G.: A signal processing approach to fair surface design. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. p. 351–358. SIGGRAPH, Association for Computing Machinery, New York, NY, USA (1995)
    Google ScholarLocate open access versionFindings
  • Thewlis, J., Albanie, S., Bilen, H., Vedaldi, A.: Unsupervised learning of landmarks by descriptor vector exchange. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 6361–6371 (2019)
    Google ScholarLocate open access versionFindings
  • Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: Flexible and deformable convolution for point clouds. In: The IEEE International Conference on Computer Vision (ICCV) (October 2019)
    Google ScholarLocate open access versionFindings
  • Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Proc. ECCV. pp. 356–369.
    Google ScholarLocate open access versionFindings
  • Varol, G., Romero, J., Martin, X., Mahmood, N., Black, M.J., Laptev, I., Schmid, C.: Learning from synthetic humans. In: CVPR (2017)
    Google ScholarFindings
  • Wang, F.D., Xue, N., Zhang, Y., Xia, G.S., Pelillo, M.: A functional representation for graph matching. IEEE transactions on pattern analysis and machine intelligence (2019)
    Google ScholarLocate open access versionFindings
  • Wang, Y., Ren, J., Yan, D.M., Guo, J., Zhang, X., Wonka, P.: Mgcn: Descriptor learning using multiscale gcns. ACM Trans. Graph. 39(4) (2020). https://doi.org/10.1145/3386569.3392443
    Locate open access versionFindings
  • Wang, Y., Solomon, J.M.: Prnet: Self-supervised learning for partial-to-partial registration. In: Advances in Neural Information Processing Systems. pp. 8812–8824 (2019)
    Google ScholarLocate open access versionFindings
  • Wei, L., Huang, Q., Ceylan, D., Vouga, E., Li, H.: Dense human body correspondences using convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1544–1553 (2016)
    Google ScholarLocate open access versionFindings
  • Zhou, J., Wang, M.J., Mao, W.D., Gong, M.L., Liu, X.P.: Siamesepointnet: A siamese point network architecture for learning 3d shape descriptor. Computer Graphics Forum 39(1), 309–321 (2020)
    Google ScholarLocate open access versionFindings
Author
Riccardo Marin
Riccardo Marin
Marie-Julie Rakotosaona
Marie-Julie Rakotosaona
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科