AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
Based on the learnt semantic part embedding via our implicit function, dense correspondence is established via the inverse function mapping from the part embedding to the corresponding 3D point

Learning Implicit Functions for Topology-Varying Dense 3D Shape Correspondence

NIPS 2020, (2020)

被引用0|浏览26
EI
下载 PDF 全文
引用
微博一下

摘要

The goal of this paper is to learn dense 3D shape correspondence for topology-varying objects in an unsupervised manner. Conventional implicit functions estimate the occupancy of a 3D point given a shape latent code. Instead, our novel implicit function produces a part embedding vector for each 3D point, which is assumed to be similar t...更多
0
简介
  • Finding dense correspondence between 3D shapes is a key algorithmic component in problems such as statistical modeling [4,5,56], cross-shape texture mapping [28], and space-time 4D reconstruction [35].
  • Man-made objects such as chairs shown in Fig. 1 are challenging to tackle, since they often differ by geometric deformations, and by part constitutions
  • In these cases, existing correspondence methods for man-made objects either perform fuzzy [27, 47] or part-level [2, 44] correspondences, or predict a constant number of semantic points [8, 19].
  • As shown in Fig. 1(c), for instance, the authors may
重点内容
  • Finding dense correspondence between 3D shapes is a key algorithmic component in problems such as statistical modeling [4,5,56], cross-shape texture mapping [28], and space-time 4D reconstruction [35]
  • The main reason is the implicit network itself is sensitive to rotation. Note that this comparison shall be viewed in the context that most baselines use extra cues during training or inference, as well as high inference speed of our learning-based approach
  • We propose a novel framework including an implicit function and its inverse for dense 3D shape correspondences of topology-varying objects
  • Based on the learnt semantic part embedding via our implicit function, dense correspondence is established via the inverse function mapping from the part embedding to the corresponding 3D point
  • Our algorithm can automatically calculate a confidence score measuring the probability of correspondence, which is desirable for man-made objects with large topological variations
  • Our proposed method provide a novel unsupervised paradigm to establish dense correspondence for topology-varying objects, which is a prerequisite for shape analysis and synthesis
方法
  • Let them first formulate the dense 3D correspondence problem. Given a collection of 3D shapes of the same object category, one may encode each shape S ∈ Rn×3 in a latent space z ∈ Rd.
  • The SEF is responsible for mapping a point from its 3D Euclidean space to the semantic embedding space.
  • If SEF could be learned for a small τ , the corresponded point q of p can be solved via q = f −1(f (p, zA), zB), where f −1(:, :) is the inverse function of f that maps a point from the semantic embedding space back to the 3D space.
  • The dense correspondence amounts to learning the SEF and its inverse function
结果
  • The correspondence accuracy is measured by the fraction of correspondences whose error is below a given threshold of Euclidean distances.
  • The authors' method achieves competitive performance as baselines
  • While it has the best AUC overall, it is worse at the threshold between [0, 0.05].
  • The main reason is the implicit network itself is sensitive to rotation
  • Note that this comparison shall be viewed in the context that most baselines use extra cues during training or inference, as well as high inference speed of the learning-based approach.
  • Note the amount of non-existent correspondence is impacted by the threshold τ as in Fig. 4(b).
  • This is expected as the division of semantically corresponded or not can be blurred for some shape parts
结论
  • The authors propose a novel framework including an implicit function and its inverse for dense 3D shape correspondences of topology-varying objects.
  • With the increasing number and diversity of 3D CAD models in online repositories, there is a growing need for leverage them to facilitate future product development due to their similarities in function and shape.
  • Towards this goal, the proposed method provide a novel unsupervised paradigm to establish dense correspondence for topology-varying objects, which is a prerequisite for shape analysis and synthesis.
  • As the approach is designed for generic objects, its application space can be extremely wide
总结
  • Introduction:

    Finding dense correspondence between 3D shapes is a key algorithmic component in problems such as statistical modeling [4,5,56], cross-shape texture mapping [28], and space-time 4D reconstruction [35].
  • Man-made objects such as chairs shown in Fig. 1 are challenging to tackle, since they often differ by geometric deformations, and by part constitutions
  • In these cases, existing correspondence methods for man-made objects either perform fuzzy [27, 47] or part-level [2, 44] correspondences, or predict a constant number of semantic points [8, 19].
  • As shown in Fig. 1(c), for instance, the authors may
  • Objectives:

    The goal of this paper is to learn dense 3D shape correspondence for topologyvarying objects in an unsupervised manner.
  • Methods:

    Let them first formulate the dense 3D correspondence problem. Given a collection of 3D shapes of the same object category, one may encode each shape S ∈ Rn×3 in a latent space z ∈ Rd.
  • The SEF is responsible for mapping a point from its 3D Euclidean space to the semantic embedding space.
  • If SEF could be learned for a small τ , the corresponded point q of p can be solved via q = f −1(f (p, zA), zB), where f −1(:, :) is the inverse function of f that maps a point from the semantic embedding space back to the 3D space.
  • The dense correspondence amounts to learning the SEF and its inverse function
  • Results:

    The correspondence accuracy is measured by the fraction of correspondences whose error is below a given threshold of Euclidean distances.
  • The authors' method achieves competitive performance as baselines
  • While it has the best AUC overall, it is worse at the threshold between [0, 0.05].
  • The main reason is the implicit network itself is sensitive to rotation
  • Note that this comparison shall be viewed in the context that most baselines use extra cues during training or inference, as well as high inference speed of the learning-based approach.
  • Note the amount of non-existent correspondence is impacted by the threshold τ as in Fig. 4(b).
  • This is expected as the division of semantically corresponded or not can be blurred for some shape parts
  • Conclusion:

    The authors propose a novel framework including an implicit function and its inverse for dense 3D shape correspondences of topology-varying objects.
  • With the increasing number and diversity of 3D CAD models in online repositories, there is a growing need for leverage them to facilitate future product development due to their similarities in function and shape.
  • Towards this goal, the proposed method provide a novel unsupervised paradigm to establish dense correspondence for topology-varying objects, which is a prerequisite for shape analysis and synthesis.
  • As the approach is designed for generic objects, its application space can be extremely wide
表格
  • Table1: Unsupervised segmentation on ShapeNet part. We use #parts in evaluation and k=12 for all 8 models
  • Table2: Stages of the training process
Download tables as Excel
相关工作
  • Dense Shape Correspondence While there are many dense correspondence works for organic shapes [6, 15, 18, 29, 30, 37, 42, 48], due to space, our review focuses on methods designed for manmade objects, including optimization and learning-based methods. For the former, most prior works build correspondences only at a part level [2,21,25,44,55]. Kim et al [27] propose a diffusion map to compute point-based “fuzzy correspondence” for every shape pair. This is only effective for a small collection of shapes with limited shape variations. [26] and [20] present a template-based deformation method, which can find point-level correspondences after rigid alignment between the template and target shapes. However, these methods only predict coarse and discrete correspondence, leaving the structural or topological discrepancies between matched parts or part ensembles unresolved.

    A series of learning-based methods [19, 34, 49, 53, 54] are proposed to learn local descriptors, and treat correspondence as 3D semantic landmark estimation. E.g., ShapeUnicode [34] learns a unified embedding for 3D shapes and demonstrates its ability in correspondence among 3D shapes. However, these methods require ground-truth pairwise correspondences for training. Recently, Chen et al [8] present an unsupervised method to estimate 3D structure points. Unfortunately, it estimates a constant number of sparse structured points. As shapes may have diverse part constitutions, it may not be meaningful to establish the correspondence between all of their points. Groueix et al [17] also learn a parametric transformation between two surfaces by leveraging cycle-consistency, and apply to the segmentation problem. However, the deformation-based method always deforms all points on one shape to another, even the points from a non-matching part. In contrast, our unsupervisedly learnt model can perform pairwise dense correspondence for any two shapes of a man-made object.
基金
  • At the distance threshold of 0.05, our method improves on average 17% accuracy in 4 categories over [8]
研究对象与分析
pairs: 9900
For testing, BHCP provides ground-truth semantic points (7-13 per shape) of 404 shapes including plane (104), bike (100), chair (100), helicopter (100). We generate all pairs of shapes for testing, e.g., 9, 900 pairs for bike. The helicopter is tested with the plane model as [8, 19] did

samples: 500
Despite unsupervisedly segmenting chairs into 3 parts, the extra 9 = (k − 3) dimensions of PEV benefit the finer-grained task of correspondence (Fig. 7(c)), which in turns help segmentation. Computation Time Our training on one category (500 samples) takes ∼8 hours to converge with a GTX1080Ti GPU, where 1, 1, and 6 hours are spent at Stage 1, 2, 3 respectively. In inference, the average runtime to pair two shapes (n=8,192) is 0.21 second including runtimes of E, f , g networks, neighbour search and confidence calculation

data: 8192
Computation Time Our training on one category (500 samples) takes ∼8 hours to converge with a GTX1080Ti GPU, where 1, 1, and 6 hours are spent at Stage 1, 2, 3 respectively. In inference, the average runtime to pair two shapes (n=8,192) is 0.21 second including runtimes of E, f , g networks, neighbour search and confidence calculation. In this work, we propose a novel framework including an implicit function and its inverse for dense 3D shape correspondences of topology-varying objects

引用论文
  • Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3D point clouds. In ICML, 2018.
    Google ScholarLocate open access versionFindings
  • Ibraheem Alhashim, Kai Xu, Yixin Zhuang, Junjie Cao, Patricio Simari, and Hao Zhang. Deformation-driven topology-varying 3D shape correspondence. TOG, 2015.
    Google ScholarLocate open access versionFindings
  • Matan Atzmon, Niv Haim, Lior Yariv, Ofer Israelov, Haggai Maron, and Yaron Lipman. Controlling neural level sets. In NeurIPS, 2019.
    Google ScholarLocate open access versionFindings
  • Volker Blanz and Thomas Vetter. Face recognition based on fitting a 3D morphable model. TPAMI, 2003.
    Google ScholarFindings
  • Federica Bogo, Javier Romero, Matthew Loper, and Michael J Black. FAUST: Dataset and evaluation for 3D mesh registration. In CVPR, 2014.
    Google ScholarLocate open access versionFindings
  • Davide Boscaini, Jonathan Masci, Emanuele Rodolà, and Michael Bronstein. Learning shape correspondence with anisotropic convolutional neural networks. In NeurIPS, 2016.
    Google ScholarLocate open access versionFindings
  • Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012, 2015.
    Findings
  • Nenglun Chen, Lingjie Liu, Zhiming Cui, Runnan Chen, Duygu Ceylan, Changhe Tu, and Wenping Wang. Unsupervised learning of intrinsic structural representation points. In CVPR, 2020.
    Google ScholarLocate open access versionFindings
  • Zhiqin Chen, Kangxue Yin, Matthew Fisher, Siddhartha Chaudhuri, and Hao Zhang. BAE-NET: branched autoencoder for shape co-segmentation. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In CVPR, 2019.
    Google ScholarLocate open access versionFindings
  • Theo Deprelle, Thibault Groueix, Matthew Fisher, Vladimir Kim, Bryan Russell, and Mathieu Aubry. Learning elementary structures for 3D shape generation and matching. In NeurIPS, 2019.
    Google ScholarLocate open access versionFindings
  • Kyle Genova, Forrester Cole, Avneesh Sud, Aaron Sarna, and Thomas Funkhouser. Local deep implicit functions for 3D shape. In CVPR, 2020.
    Google ScholarLocate open access versionFindings
  • Kyle Genova, Forrester Cole, Daniel Vlasic, Aaron Sarna, William T Freeman, and Thomas Funkhouser. Learning shape templates with structured implicit functions. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Justin Johnson Georgia Gkioxari, Jitendra Malik. Mesh R-CNN. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. D-CODED: 3D correspondences by deep deformation. In ECCV, 2018.
    Google ScholarLocate open access versionFindings
  • Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. Atlasnet: A papier-mâché approach to learning 3D surface generation. In CVPR, 2018.
    Google ScholarLocate open access versionFindings
  • Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. Unsupervised cycle-consistent deformation for shape matching. In Computer Graphics Forum, 2019.
    Google ScholarLocate open access versionFindings
  • Oshri Halimi, Or Litany, Emanuele Rodola, Alex M Bronstein, and Ron Kimmel. Unsupervised learning of dense shape correspondence. In CVPR, 2019.
    Google ScholarLocate open access versionFindings
  • Haibin Huang, Evangelos Kalogerakis, Siddhartha Chaudhuri, Duygu Ceylan, Vladimir G Kim, and Ersin Yumer. Learning local shape descriptors from part correspondences with multiview convolutional networks. TOG, 2017.
    Google ScholarLocate open access versionFindings
  • Haibin Huang, Evangelos Kalogerakis, and Benjamin Marlin. Analysis and synthesis of 3D shape families via deep-learned generative models of surfaces. In Computer Graphics Forum, 2015.
    Google ScholarLocate open access versionFindings
  • Qixing Huang, Vladlen Koltun, and Leonidas Guibas. Joint shape segmentation with linear programming. In SIGGRAPH Asia, 2011.
    Google ScholarLocate open access versionFindings
  • Qixing Huang, Fan Wang, and Leonidas Guibas. Functional map networks for analyzing and exploring large shape collections. TOG, 2014.
    Google ScholarLocate open access versionFindings
  • Xiaolei Huang, Nikos Paragios, and Dimitris N Metaxas. Shape registration in implicit spaces using information theory and free form deformations. TPAMI, 2006.
    Google ScholarLocate open access versionFindings
  • Xiaolei Huang, Song Zhang, Yang Wang, Dimitris Metaxas, and Dimitris Samaras. A hierarchical framework for high resolution facial expression tracking. In CVPRW, 2004.
    Google ScholarLocate open access versionFindings
  • Evangelos Kalogerakis, Aaron Hertzmann, and Karan Singh. Learning 3D mesh segmentation and labeling. In SIGGRAPH, 2010.
    Google ScholarLocate open access versionFindings
  • Vladimir G Kim, Wilmot Li, Niloy J Mitra, Siddhartha Chaudhuri, Stephen DiVerdi, and Thomas Funkhouser. Learning part-based templates from large collections of 3D shapes. TOG, 2013.
    Google ScholarLocate open access versionFindings
  • Vladimir G Kim, Wilmot Li, Niloy J Mitra, Stephen DiVerdi, and Thomas Funkhouser. Exploring collections of 3D models using fuzzy correspondences. TOG, 2012.
    Google ScholarLocate open access versionFindings
  • Vladislav Kraevoy, Alla Sheffer, and Craig Gotsman. Matchmaker: constructing constrained texture maps. TOG, 2003.
    Google ScholarFindings
  • Sing Chun Lee and Misha Kazhdan. Dense point-to-point correspondences between genus-zero shapes. In Computer Graphics Forum, 2019.
    Google ScholarLocate open access versionFindings
  • Or Litany, Tal Remez, Emanuele Rodolà, Alex Bronstein, and Michael Bronstein. Deep functional maps: Structured prediction for dense shape correspondence. In ICCV, 2017.
    Google ScholarLocate open access versionFindings
  • Feng Liu, Luan Tran, and Xiaoming Liu. 3D face modeling from diverse raw scan data. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Shichen Liu, Shunsuke Saito, Weikai Chen, and Hao Li. Learning to infer implicit surfaces without 3D supervision. In NeurIPS, 2019.
    Google ScholarLocate open access versionFindings
  • Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3D reconstruction in function space. In CVPR, 2019.
    Google ScholarLocate open access versionFindings
  • Sanjeev Muralikrishnan, Vladimir G Kim, Matthew Fisher, and Siddhartha Chaudhuri. Shape unicode: A unified shape representation. In CVPR, 2019.
    Google ScholarLocate open access versionFindings
  • Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Occupancy flow: 4D reconstruction by learning particle dynamics. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Michael Oechsle, Lars Mescheder, Michael Niemeyer, Thilo Strauss, and Andreas Geiger. Texture fields: Learning texture representations in function space. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Functional maps: a flexible representation of maps between shapes. TOG, 2012.
    Google ScholarLocate open access versionFindings
  • Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. In CVPR, 2019.
    Google ScholarLocate open access versionFindings
  • Despoina Paschalidou, Luc van Gool, and Andreas Geiger. Learning unsupervised hierarchical part decomposition of 3D objects from a single rgb image. In CVPR, 2020.
    Google ScholarLocate open access versionFindings
  • Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3D classification and segmentation. In CVPR, 2017.
    Google ScholarLocate open access versionFindings
  • Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In NeurIPS, 2017.
    Google ScholarLocate open access versionFindings
  • Jean-Michel Roufosse, Abhishek Sharma, and Maks Ovsjanikov. Unsupervised deep learning for structured shape matching. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. PIFu: Pixel-aligned implicit function for high-resolution clothed human digitization. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Oana Sidi, Oliver van Kaick, Yanir Kleiman, Hao Zhang, and Daniel Cohen-Or. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. In SIGGRAPH Asia, 2011.
    Google ScholarLocate open access versionFindings
  • Vincent Sitzmann, Michael Zollhöfer, and Gordon Wetzstein. Scene representation networks: Continuous 3D-structure-aware neural scene representations. In NeurIPS, 2019.
    Google ScholarLocate open access versionFindings
  • Miroslava Slavcheva, Maximilian Baust, and Slobodan Ilic. Towards implicit correspondence in signed distance field evolution. In ICCV, 2017.
    Google ScholarLocate open access versionFindings
  • Justin Solomon, Andy Nguyen, Adrian Butscher, Mirela Ben-Chen, and Leonidas Guibas. Soft maps between surfaces. In Computer Graphics Forum, 2012.
    Google ScholarLocate open access versionFindings
  • Florian Steinke, Volker Blanz, and Bernhard Schölkopf. Learning dense 3D correspondence. In NeurIPS, 2007.
    Google ScholarLocate open access versionFindings
  • Minhyuk Sung, Hao Su, Ronald Yu, and Leonidas J Guibas. Deep functional dictionaries: Learning consistent semantic structures on 3D models from functions. In NeurIPS, 2018.
    Google ScholarLocate open access versionFindings
  • Oliver Van Kaick, Hao Zhang, Ghassan Hamarneh, and Daniel Cohen-Or. A survey on shape correspondence. In Computer Graphics Forum, 2011.
    Google ScholarLocate open access versionFindings
  • Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3D mesh models from single RGB images. In ECCV, 2018.
    Google ScholarLocate open access versionFindings
  • Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3D shape collections. TOG, 2016.
    Google ScholarLocate open access versionFindings
  • Li Yi, Hao Su, Xingwen Guo, and Leonidas J Guibas. SyncspecCNN: Synchronized spectral CNN for 3D shape segmentation. In CVPR, 2017.
    Google ScholarLocate open access versionFindings
  • Yang You, Yujing Lou, Chengkun Li, Zhoujun Cheng, Liangwei Li, Lizhuang Ma, Cewu Lu, and Weiming Wang. Keypointnet: A large-scale 3D keypoint dataset aggregated from numerous human annotations. In CVPR, 2020.
    Google ScholarLocate open access versionFindings
  • Chenyang Zhu, Renjiao Yi, Wallace Lira, Ibraheem Alhashim, Kai Xu, and Hao Zhang. Deformation-driven shape correspondence via shape recognition. TOG, 2017.
    Google ScholarLocate open access versionFindings
  • Silvia Zuffi, Angjoo Kanazawa, David W Jacobs, and Michael J Black. 3D menagerie: Modeling the 3D shape and pose of animals. In CVPR, 2017.
    Google ScholarLocate open access versionFindings
您的评分 :
0

 

标签
评论
小科