Philipp ErlerORCID iD, Paul Guerrero, Stefan Ohrhallinger, Michael WimmerORCID iD, Niloy Mitra
Points2Surf: Learning Implicit Surfaces from Point Clouds
In Computer Vision -- ECCV 2020, pages 108-124. October 2020.
[points2surf_paper] [short video]

Information

Abstract

A key step in any scanning-based asset creation workflow is to convert unordered point clouds to a surface. Classical methods (e.g., Poisson reconstruction) start to degrade in the presence of noisy and partial scans. Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans. However, such data-driven methods struggle to generalize to new shapes with large geometric and topological variations. We present Points2Surf, a novel patch-based learning framework that produces accurate surfaces directly from raw scans without normals.

Learning a prior over a combination of detailed local patches and coarse global information improves generalization performance and reconstruction accuracy.

Our extensive comparison on both synthetic and real data demonstrates a clear advantage of our method over state-of-the-art alternatives on previously unseen classes (on average, Points2Surf brings down reconstruction error by 30% over SPR and by 270%+ over deep learning based SotA methods) at the cost of longer computation times and a slight increase in small-scale topological noise in some cases. Our source code, pre-trained model, and dataset are available on: https://github.com/ErlerPhilipp/points2surf

Additional Files and Images

Additional images and videos

long video: 8 min video for ECCV 2020 long video: 8 min video for ECCV 2020
short video: 2 min video for ECCV 2020 short video: 2 min video for ECCV 2020
teaser: We present Points2Surf, a method to reconstruct an accurate implicit surface from a noisy point cloud. Unlike current data-driven surface reconstruction methods like DeepSDF and AtlasNet, it is patch-based, improves detail reconstruction, and unlike Screened Poisson Reconstruction (SPR), a learned prior of low-level patch shapes improves reconstruction accuracy. Note the quality of reconstructions, both geometric and topological, against the original surfaces. The ability of Points2Surf to generalize to new shapes makes it the first learning-based approach with significant generalization ability under both geometric and topological variations. teaser: We present Points2Surf, a method to reconstruct an accurate implicit surface from a noisy point cloud. Unlike current data-driven surface reconstruction methods like DeepSDF and AtlasNet, it is patch-based, improves detail reconstruction, and unlike Screened Poisson Reconstruction (SPR), a learned prior of low-level patch shapes improves reconstruction accuracy. Note the quality of reconstructions, both geometric and topological, against the original surfaces. The ability of Points2Surf to generalize to new shapes makes it the first learning-based approach with significant generalization ability under both geometric and topological variations.

Additional files

abc_meshes: Our normalized ground-truth meshes from the ABC dataset. abc_meshes: Our normalized ground-truth meshes from the ABC dataset.
abc_training: Our processed extract from the ABC dataset that can be used to train Points2Surf. abc_training: Our processed extract from the ABC dataset that can be used to train Points2Surf.
abc: Our processed extract from the ABC dataset that can be used to replicate the results of Points2Surf. abc: Our processed extract from the ABC dataset that can be used to replicate the results of Points2Surf.
ablation_models: Our pre-trained models (ablation versions) to reproduce the results. ablation_models: Our pre-trained models (ablation versions) to reproduce the results.
famous: Our processed Famous dataset that can be used to replicate the results of Points2Surf. famous: Our processed Famous dataset that can be used to replicate the results of Points2Surf.
max_model: A new model version with even better results. max_model: A new model version with even better results.
points2surf_paper: Points2Surf paper (Arxiv version) points2surf_paper: Points2Surf paper (Arxiv version)
real_world: Our processed real-world dataset that can be used to replicate the results of Points2Surf. real_world: Our processed real-world dataset that can be used to replicate the results of Points2Surf.
thingi10k: Our processed extract from the Thingi10k dataset that can be used to replicate the results of Points2Surf. thingi10k: Our processed extract from the Thingi10k dataset that can be used to replicate the results of Points2Surf.
vanilla_model: Our pre-trained model (vanilla version) to reproduce the results. vanilla_model: Our pre-trained model (vanilla version) to reproduce the results.

Weblinks

BibTeX

@inproceedings{erler-2020-p2s,
  title =      "Points2Surf: Learning Implicit Surfaces from Point Clouds",
  author =     "Philipp Erler and Paul Guerrero and Stefan Ohrhallinger and
               Michael Wimmer and Niloy Mitra",
  year =       "2020",
  abstract =   "A key step in any scanning-based asset creation workflow is
               to convert unordered point clouds to a surface. Classical
               methods (e.g., Poisson reconstruction) start to degrade in
               the presence of noisy and partial scans. Hence, deep
               learning based methods have recently been proposed to
               produce complete surfaces, even from partial scans. However,
               such data-driven methods struggle to generalize to new
               shapes with large geometric and topological variations. We
               present Points2Surf, a novel patch-based learning framework
               that produces accurate surfaces directly from raw scans
               without normals.  Learning a prior over a combination of
               detailed local patches and coarse global information
               improves generalization performance and reconstruction
               accuracy.  Our extensive comparison on both synthetic and
               real data demonstrates a clear advantage of our method over
               state-of-the-art alternatives on previously unseen classes
               (on average, Points2Surf brings down reconstruction error by
               30% over SPR and by 270%+ over deep learning based SotA
               methods) at the cost of longer computation times and a
               slight increase in small-scale topological noise in some
               cases.  Our source code, pre-trained model, and dataset are
               available on: https://github.com/ErlerPhilipp/points2surf ",
  month =      oct,
  isbn =       "978-3-030-58558-7",
  series =     "Lecture Notes in Computer Science",
  publisher =  "Springer International Publishing",
  location =   "Glasgow, UK (online)",
  address =    "Cham",
  event =      "ECCV 2020",
  editor =     "Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and
               Frahm, Jan-Michael",
  doi =        "10.1007/978-3-030-58558-7_7",
  booktitle =  "Computer Vision -- ECCV 2020",
  journal =    "Computer Vision – ECCV 2020",
  pages =      "17",
  volume =     "12350",
  pages =      "108--124",
  keywords =   "surface reconstruction, implicit surfaces, point clouds,
               patch-based, local and global, deep learning, generalization",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2020/erler-2020-p2s/",
}