Points2Surf: Learning Implicit Surfaces from Point Clouds | TU Wien

Information

Publication Type: Conference Paper
Workgroup(s)/Project(s):
- Superhumans - Walking Through Walls
- MAKE-IT-FAB: Modeling of Shapes for Personal Fabrication
- Real-Time Shape Acquisition with Sensor-Specific Precision
Date: October 2020
ISBN: 978-3-030-58558-7
Call for Papers: Call for Paper
Volume: 12350
Journal: Computer Vision – ECCV 2020
Conference date: 24. August 2020 – 27. August 2020
Booktitle: Computer Vision -- ECCV 2020
DOI: 10.1007/978-3-030-58558-7_7
Series: Lecture Notes in Computer Science
Editor: Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael
Event: ECCV 2020
Address: Cham
Lecturer: Philipp Erler
Location: Glasgow, UK (online)
Open Access: yes
Publisher: Springer International Publishing
Pages: 108 – 124
Keywords: surface reconstruction, implicit surfaces, point clouds, patch-based, local and global, deep learning, generalization

Abstract

A key step in any scanning-based asset creation workflow is to convert unordered point clouds to a surface. Classical methods (e.g., Poisson reconstruction) start to degrade in the presence of noisy and partial scans. Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans. However, such data-driven methods struggle to generalize to new shapes with large geometric and topological variations. We present Points2Surf, a novel patch-based learning framework that produces accurate surfaces directly from raw scans without normals.

Learning a prior over a combination of detailed local patches and coarse global information improves generalization performance and reconstruction accuracy.

Our extensive comparison on both synthetic and real data demonstrates a clear advantage of our method over state-of-the-art alternatives on previously unseen classes (on average, Points2Surf brings down reconstruction error by 30% over SPR and by 270%+ over deep learning based SotA methods) at the cost of longer computation times and a slight increase in small-scale topological noise in some cases. Our source code, pre-trained model, and dataset are available on: https://github.com/ErlerPhilipp/points2surf

Additional Files and Images

Additional images and videos

long video: 8 min video for ECCV 2020

short video: 2 min video for ECCV 2020

teaser: We present Points2Surf, a method to reconstruct an accurate implicit surface from a noisy point cloud. Unlike current data-driven surface reconstruction methods like DeepSDF and AtlasNet, it is patch-based, improves detail reconstruction, and unlike Screened Poisson Reconstruction (SPR), a learned prior of low-level patch shapes improves reconstruction accuracy. Note the quality of reconstructions, both geometric and topological, against the original surfaces. The ability of Points2Surf to generalize to new shapes makes it the first learning-based approach with significant generalization ability under both geometric and topological variations.

Additional files

abc_meshes: Our normalized ground-truth meshes from the ABC dataset.

abc_training: Our processed extract from the ABC dataset that can be used to train Points2Surf.

abc: Our processed extract from the ABC dataset that can be used to replicate the results of Points2Surf.

ablation_models: Our pre-trained models (ablation versions) to reproduce the results.

famous: Our processed Famous dataset that can be used to replicate the results of Points2Surf.

max_model: A new model version with even better results.

points2surf_paper: Points2Surf paper (Arxiv version)

real_world: Our processed real-world dataset that can be used to replicate the results of Points2Surf.

thingi10k: Our processed extract from the Thingi10k dataset that can be used to replicate the results of Points2Surf.

vanilla_model: Our pre-trained model (vanilla version) to reproduce the results.

Weblinks

Github Repo
Access the source code, pre-trained models and datasets from Github.
Arxiv Pre-Print
Access the paper pre-print on arXiv.
Official Publication
Access the paper Springer Link.
DOI: 10.1007/978-3-030-58558-7_7

BibTeX

@inproceedings{erler-2020-p2s,
  title =      "Points2Surf: Learning Implicit Surfaces from Point Clouds",
  author =     "Philipp Erler and Paul Guerrero and Stefan Ohrhallinger and
               Michael Wimmer and Niloy Mitra",
  year =       "2020",
  abstract =   "A key step in any scanning-based asset creation workflow is
               to convert unordered point clouds to a surface. Classical
               methods (e.g., Poisson reconstruction) start to degrade in
               the presence of noisy and partial scans. Hence, deep
               learning based methods have recently been proposed to
               produce complete surfaces, even from partial scans. However,
               such data-driven methods struggle to generalize to new
               shapes with large geometric and topological variations. We
               present Points2Surf, a novel patch-based learning framework
               that produces accurate surfaces directly from raw scans
               without normals.  Learning a prior over a combination of
               detailed local patches and coarse global information
               improves generalization performance and reconstruction
               accuracy.  Our extensive comparison on both synthetic and
               real data demonstrates a clear advantage of our method over
               state-of-the-art alternatives on previously unseen classes
               (on average, Points2Surf brings down reconstruction error by
               30% over SPR and by 270%+ over deep learning based SotA
               methods) at the cost of longer computation times and a
               slight increase in small-scale topological noise in some
               cases.  Our source code, pre-trained model, and dataset are
               available on: https://github.com/ErlerPhilipp/points2surf ",
  month =      oct,
  isbn =       "978-3-030-58558-7",
  doi =        "10.1007/978-3-030-58558-7_7",
  volume =     "12350",
  journal =    "Computer Vision – ECCV 2020",
  booktitle =  "Computer Vision -- ECCV 2020",
  editor =     "Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and
               Frahm, Jan-Michael",
  series =     "Lecture Notes in Computer Science",
  event =      "ECCV 2020",
  address =    "Cham",
  location =   "Glasgow, UK (online)",
  publisher =  "Springer International Publishing",
  pages =      "108--124",
  keywords =   "surface reconstruction, implicit surfaces, point clouds,
               patch-based, local and global, deep learning, generalization",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2020/erler-2020-p2s/",
}