AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We show that the reconstruction quality by BSP-Net is competitive with state-of-the-art methods while using much fewer primitives

BSP-Net: Generating Compact Meshes via Binary Space Partitioning

CVPR, pp.42-51, (2020)

Cited by: 38|Views615
EI

Abstract

Polygonal meshes are ubiquitous in the digital 3D domain, yet they have only played a minor role in the deep learning revolution. Leading methods for learning generative models of shapes rely on implicit functions, and generate meshes only after expensive iso-surfacing routines. To overcome these challenges, we are inspired by a classic...More

Code:

Data:

0
Introduction
  • There has been an increasing interest in representation learning and generative modeling for 3D shapes.
  • BSP-Net learns an implicit field: given n point coordinates and a shape feature vector as input, the network outputs values indicating whether the points are inside or outside the shape.
  • BSP-Net is trained to reconstruct all shapes from the training set using the same set of convexes constructed in layer L2 of the network.
Highlights
  • There has been an increasing interest in representation learning and generative modeling for 3D shapes
  • Patch-based approaches can generate results which cover a 3D shape with planar polygons [46] or curved [16] mesh patches, but their visual quality is often tampered by visible seams, incoherent patch connections, and rough surface appearance
  • As much of the semantics of 3D models is captured by their surface, the boundary between inside/outside space, a variety of methods have been proposed to represent shape surfaces in a differentiable way. Amongst these we find a category of techniques pioneered by PointNet [33] that express surfaces as point clouds [1, 9, 10, 33, 34, 53, 56], and techniques pioneered by AtlasNet [16] that adopt a 2D-to-3D mapping process [47, 39, 44, 53]
  • 1While typical Binary Space Partitioning-trees are binary, we focus on n-ary trees, with the “B” in Binary Space Partitioning referring to binary space partitioning, not the tree structure
  • We study the behavior of Binary Space Partitioning-Net on a synthetic 2D shape dataset (Section 4.1), and evaluate our auto-encoder (Section 4.2), as well as single view reconstruction (Section 4.3) compared to other state-of-the-art methods
  • Note that for the segmentation task, we evaluate on Branched Auto Encoders*, the version of Branched Auto Encoders that uses the values of the predicted implicit function, and not just the classification boundaries – please note that the surface reconstructed by Branched Auto Encoders and Branched Auto Encoders* are identical
Results
  • BSP-Net is the first deep generative network which directly outputs compact and watertight polygonal meshes with arbitrary topology and structure variety.
  • By adjusting the encoder of the network, BSP-Net can be adapted for shape auto-encoding and single-view 3D reconstruction (SVR).
  • Through extensive experiments on shape auto-encoding, segmentation, part correspondence, and single-view reconstruction, the authors demonstrate state-of-the-art performances by BSP-Net. Comparisons are made to leading methods on shape decomposition and 3D reconstruction, using conventional distortion metrics, visual similarity, as well as a new metric assessing the capacity of a model in representing sharp features.
  • This work only produces a box arrangement; it does not reconstruct a structured shape like BSP-Net. Binary and capsule networks.
  • The authors study the behavior of BSP-Net on a synthetic 2D shape dataset (Section 4.1), and evaluate the auto-encoder (Section 4.2), as well as single view reconstruction (Section 4.3) compared to other state-of-the-art methods.
  • Since all these methods target shape decomposition tasks, the authors train single class networks, and evaluate segmentation as well as reconstruction performance.
  • BSP-Net achieves significantly better reconstruction quality, while maintaining high segmentation accuracy; see Table 1 and Figure 7, where the authors color each primitive based on its inferred part label.
  • The authors first compute an “edge sampling” of the surface by generating 16k points S={si} uniformly distributed on the surface of a model, and compute sharpness as: Figure 8: Single-view 3D reconstruction – comparison to AtlasNet [16], IM-NET [5], and OccNet [28].
  • Note that the method is the only one amongst those tested capable of representing sharp edges – this can be observed quantitatively in terms of Edge Chamfer Distance, where BSP-Net performs much better.
Conclusion
  • The authors introduce BSP-Net, an unsupervised method which can generate compact and structured polygonal meshes in the form of convex decomposition.
  • The authors' network learns a BSP-tree built on the same set of planes, and in turn, the same set of convexes, to minimize a reconstruction loss for the training shapes.
  • Compared to state-ofthe-art methods, meshes generated by BSP-Net exhibit superior visual quality, in particular, sharp geometric details, when comparable number of primitives are employed.
Tables
  • Table1: Surface reconstruction quality and comparison for 3D shape autoencoding. Best results are marked in bold
  • Table2: Segmentation: comparison in per-label IoU
  • Table3: Single view reconstruction – comparison to the state of the art. Atlas25 denotes AtlasNet with 25 square patches, while Atlas0 uses a single spherical patch. Subscripts to OccNet and IM-NET show sampling resolution. For fair comparisons, we use resolution 323 so that OccNet and IM-NET output meshes with comparable number of vertices and faces
  • Table4: Low-poly analysis – the dataset-averaged metrics in single view reconstruction. We highlight the number of vertices #V and triangles #F in the predicted meshes
Download tables as Excel
Related work
  • Large shape collections such as ShapeNet [2] and PartNet [30] have spurred the development of learning techniques for 3D data processing. In this section, we cover representative approaches based on the underlying shape representation learned, with a focus on generative models.

    Grid models. Early approaches generalized 2D convolutions to 3D [6, 14, 24, 48, 49], and employed volumetric grids to represent shapes in terms of coarse occupancy functions, where a voxel evaluates to zero if it is outside and one otherwise. Unfortunately, these methods are typically limited to low resolutions of at most 643 due to the cubic growth in memory requirements. To generate finer results, differentiable marching cubes operations have been proposed [26], as well as hierarchical strategies [18, 36, 42, 45, 46] that alleviate the curse of dimensionality affecting dense volumetric grids. Another alternative is to use multi-view images [25, 40] and geometry images [38, 39], which allow standard 2D convolution, but such methods are only suitable on the encoder side of a network architecture, while we focus on decoders. Finally, recent methods that perform sparse convolutions [15] on voxel grids are similarly limited to encoders.
Funding
  • This work was supported in part by an NSERC grant (No 611370), a Google Faculty Research Award, and Google Cloud Platform research credits
Reference
  • Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas J Guibas. Learning representations and generative models for 3d point clouds. 2018. 1, 3
    Google ScholarFindings
  • Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012, 2015. 3, 6, 7, 8
    Findings
  • Ding-Yun Chen, Xiao-Pei Tian, Yu-Te Shen, and Ming Ouhyoung. On visual similarity based 3d model retrieval. In Computer graphics forum, 2006
    Google ScholarFindings
  • Zhiqin Chen, Kangxue Yin, Matthew Fisher, Siddhartha Chaudhuri, and Hao Zhang. BAE-NET: Branched autoencoder for shape co-segmentation. 2019. 3, 6, 7
    Google ScholarFindings
  • Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. 2019. 1, 2, 3, 4, 5, 7
    Google ScholarFindings
  • Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. 2013, 7
    Google ScholarFindings
  • Angela Dai and Matthias Nießner. Scan2mesh: From unstructured range scans to 3d meshes. 2019. 1, 3
    Google ScholarFindings
  • Boyang Deng, Kyle Genova, Soroosh Yazdani, Sofien Bouaziz, Geoffrey Hinton, and Andrea Tagliasacchi. Cvxnets: Learnable convex decomposition. 2020. 3
    Google ScholarFindings
  • Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a single image. 2017. 3
    Google ScholarFindings
  • Matheus Gadelha, Rui Wang, and Subhransu Maji. Multiresolution tree networks for 3d point cloud processing. 2018. 3
    Google ScholarFindings
  • Lin Gao, Yu-Kun Lai, Jie Yang, Ling-Xiao Zhang, Leif Kobbelt, and Shihong Xia. Sparse data driven mesh deformation. IEEE transactions on visualization and computer graphics, 2019. 1
    Google ScholarLocate open access versionFindings
  • Lin Gao, Jie Yang, Tong Wu, Hongbo Fu, Yu-Kun Lai, and Hao Zhang. SDM-NET: Deep generative network for structured deformable mesh. ACM Trans. Gr., 2019. 1, 3
    Google ScholarLocate open access versionFindings
  • Kyle Genova, Forrester Cole, Daniel Vlasic, Aaron Sarna, William T. Freeman, and Thomas Funkhouser. Learning shape templates with structured implicit functions. 2019. 1, 3
    Google ScholarFindings
  • Rohit Girdhar, David F Fouhey, Mikel Rodriguez, and Abhinav Gupta. Learning a predictable and generative vector representation for objects. 2016. 1, 3
    Google ScholarFindings
  • Benjamin Graham, Martin Engelcke, and Laurens van der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. 2018. 3
    Google ScholarFindings
  • Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan Russell, and Mathieu Aubry. Atlasnet: A papier-macheapproach to learning 3d surface generation. 2018. 1, 3, 7
    Google ScholarFindings
  • Heli Ben Hamu, Haggai Maron, Itay Kezurer, Gal Avineri, and Yaron Lipman. Multi-chart generative surface modeling. ACM Trans. Gr., 2018. 1
    Google ScholarLocate open access versionFindings
  • Christian Hane, Shubham Tulsiani, and Jitendra Malik. Hierarchical surface prediction for 3d object reconstruction. 2017. 1, 3
    Google ScholarFindings
  • Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. MeshCNN: A network with an edge. ACM Trans. Gr., 201
    Google ScholarLocate open access versionFindings
  • Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 1991. 3
    Google ScholarLocate open access versionFindings
  • Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran ElYaniv, and Yoshua Bengio. Binarized neural networks. 2016. 3
    Google ScholarFindings
  • Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. 2019. 1, 2, 3, 8
    Google ScholarFindings
  • Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas Guibas. Grass: Generative recursive autoencoders for shape structures. ACM Trans. Gr., 2017. 3
    Google ScholarLocate open access versionFindings
  • Yiyi Liao, Simon Donne, and Andreas Geiger. Deep marching cubes: Learning explicit surface representations. 2018. 3
    Google ScholarFindings
  • Chen-Hsuan Lin, Chen Kong, and Simon Lucey. Learning efficient point cloud generation for dense 3d object reconstruction. In AAAI Conference on Artificial Intelligence (AAAI), 2018. 3
    Google ScholarLocate open access versionFindings
  • William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3d surface construction algorithm. 1987. 1, 3
    Google ScholarFindings
  • Haggai Maron, Meirav Galun, Noam Aigerman, Miri Trope, Nadav Dym, Ersin Yumer, Vladimir G. Kim, and Yaron Lipman. Convolutional neural networks on surfaces via seamless toric covers. ACM Trans. Gr., 2017. 1
    Google ScholarLocate open access versionFindings
  • Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3D reconstruction in function space. 2019. 1, 2, 3, 7
    Google ScholarFindings
  • Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, and Leonidas J Guibas. Structurenet: Hierarchical graph networks for 3d shape generation. 2019. 3
    Google ScholarFindings
  • Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Leonidas J. Guibas, and Hao Su. PartNet: A largescale benchmark for fine-grained and hierarchical part-level 3D object understanding. 2019. 3
    Google ScholarFindings
  • Chengjie Niu, Jun Li, and Kai Xu. Im2struct: Recovering 3d shape structure from a single RGB image. 2018. 3
    Google ScholarFindings
  • Despoina Paschalidou, Ali Osman Ulusoy, and Andreas Geiger. Superquadrics revisited: Learning 3d shape parsing beyond cuboids. 2019. 3, 6
    Google ScholarFindings
  • Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep learning on point sets for 3D classification and segmentation. 2017. 1, 3
    Google ScholarFindings
  • Charles R. Qi, Li Yi, Hao Su, and Leonidas J Guibas. PointNet++: Deep hierarchical feature learning on point sets in a metric space. 2017. 1, 3
    Google ScholarFindings
  • Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. 2016. 3
    Google ScholarFindings
  • Gernot Riegler, Ali Osman Ulusoy, Horst Bischof, and Andreas Geiger. Octnetfusion: Learning depth fusion from data. 2017. 3
    Google ScholarFindings
  • Sara Sabour, Nicholas Frosst, and Geoffrey E. Hinton. Dynamic routing between capsules. 2017. 3
    Google ScholarFindings
  • Ayan Sinha, Jing Bai, and Karthik Ramani. Deep learning 3d shape surfaces using geometry images. 2016. 3
    Google ScholarFindings
  • Ayan Sinha, Asim Unmesh, Qixing Huang, and Karthik Ramani. Surfnet: Generating 3d shape surfaces using deep residual networks. 2017. 3
    Google ScholarFindings
  • Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. Multi-view convolutional neural networks for 3d shape recognition. 2015. 3
    Google ScholarFindings
  • Qingyang Tan, Lin Gao, Yu-Kun Lai, and Shihong Xia. Variational autoencoders for deforming 3D mesh models. 2018. 1
    Google ScholarFindings
  • Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs. 2017. 3
    Google ScholarFindings
  • Shubham Tulsiani, Hao Su, Leonidas J Guibas, Alexei A Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives. 2017. 3, 6
    Google ScholarFindings
  • Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. 2018. 1, 3
    Google ScholarFindings
  • Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Sun, and Xin Tong. O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis. 2017. 3
    Google ScholarFindings
  • Peng-Shuai Wang, Chun-Yu Sun, Yang Liu, and Xin Tong. Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapes. 2018. 1, 3
    Google ScholarFindings
  • Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. Deep geometric prior for surface reconstruction. 2019. 3
    Google ScholarFindings
  • Jiajun Wu, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. 2016. 1, 3
    Google ScholarFindings
  • Jiajun Wu, Chengkai Zhang, Xiuming Zhang, Zhoutong Zhang, William T. Freeman, and Joshua B. Tenenbaum. Learning shape priors for single-view 3d completion and reconstruction. 2018. 3
    Google ScholarFindings
  • Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. 2015. 1
    Google ScholarFindings
  • Zhijie Wu, Xiang Wang, Di Lin, Dani Lischinski, Daniel Cohen-Or, and Hui Huang. SAGNet: Structure-aware generative network for 3D-shape modeling. 2019. 3
    Google ScholarFindings
  • Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomır Mech, and Ulrich Neumann. DISN: deep implicit surface network for high-quality single-view 3d reconstruction. 2019. 1
    Google ScholarFindings
  • Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. 2018. 3
    Google ScholarFindings
  • Li Yi, Vladimir G Kim, Duygu Ceylan, I Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, Leonidas Guibas, et al. A scalable active framework for region annotation in 3D shape collections. 35(6), 2016. 6
    Google ScholarFindings
  • Kangxue Yin, Zhiqin Chen, Hui Huang, Daniel Cohen-Or, and Hao Zhang. LOGAN: Unpaired shape transform in latent overcomplete space. ACM Trans. Gr., 2019. 1
    Google ScholarLocate open access versionFindings
  • Kangxue Yin, Hui Huang, Daniel Cohen-Or, and Hao Zhang. P2P-NET: Bidirectional point displacement net for shape transform. ACM Trans. Gr., 2018. 1, 3
    Google ScholarLocate open access versionFindings
  • Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Renjiao Yi, and Hao Zhang. SCORES: Shape composition with recursive substructure priors. ACM Trans. Gr., 2018. 3
    Google ScholarLocate open access versionFindings
  • Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem. 3D-PRNN: Generating shape primitives with recurrent neural networks. 2017. 3
    Google ScholarFindings
Your rating :
0

 

Best Paper
Best Paper of CVPR, 2019
Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科