A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets | TU Wien

Information

Publication Type: Journal Paper with Conference Talk
Workgroup(s)/Project(s):
- Instant Visualization and Interaction for Large Point Clouds
Date: July 2024
Journal: ACM Transactions on Graphics
Volume: 43
Number: 4
Location: Denver, USA
Lecturer: Andréas Meuleman
Article Number: 62
ISSN: 1557-7368
Event: ACM SIGGRAPH 2024
DOI: 10.1145/3658160
Pages: 15
Publisher: ASSOC COMPUTING MACHINERY
Conference date: 28. July 2024 – 1. August 2024
Pages: 1 – 15
Keywords: real-time rendering, 3d gaussian splatting, level-of-detail, Large Scenes

Abstract

Novel view synthesis has seen major advances in recent years, with 3D Gaussian splatting offering an excellent level of visual quality, fast training and real-time rendering. However, the resources needed for training and rendering inevitably limit the size of the captured scenes that can be represented with good visual quality. We introduce a hierarchy of 3D Gaussians that preserves visual quality for very large scenes, while offering an efficient Level-of-Detail (LOD) solution for efficient rendering of distant content with effective level selection and smooth transitions between levels. We introduce a divide-and-conquer approach that allows us to train very large scenes in independent chunks. We consolidate the chunks into a hierarchy that can be optimized to further improve visual quality of Gaussians merged into intermediate nodes. Very large captures typically have sparse coverage of the scene, presenting many challenges to the original 3D Gaussian splatting training method; we adapt and regularize training to account for these issues. We present a complete solution, that enables real-time rendering of very large scenes and can adapt to available resources thanks to our LOD method. We show results for captured scenes with up to tens of thousands of images with a simple and affordable rig, covering trajectories of up to several kilometers and lasting up to one hour.

Additional Files and Images

paper

Weblinks

BibTeX

@article{kerbl-2024-ah3,
  title =      "A Hierarchical 3D Gaussian Representation for Real-Time
               Rendering of Very Large Datasets",
  author =     "Bernhard Kerbl and Andr\'{e}as Meuleman and Georgios Kopanas
               and Michael Wimmer and Alexandre Lanvin and G. Drettakis",
  year =       "2024",
  abstract =   "Novel view synthesis has seen major advances in recent
               years, with 3D Gaussian splatting offering an excellent
               level of visual quality, fast training and real-time
               rendering. However, the resources needed for training and
               rendering inevitably limit the size of the captured scenes
               that can be represented with good visual quality. We
               introduce a hierarchy of 3D Gaussians that preserves visual
               quality for very large scenes, while offering an efficient
               Level-of-Detail (LOD) solution for efficient rendering of
               distant content with effective level selection and smooth
               transitions between levels. We introduce a
               divide-and-conquer approach that allows us to train very
               large scenes in independent chunks. We consolidate the
               chunks into a hierarchy that can be optimized to further
               improve visual quality of Gaussians merged into intermediate
               nodes. Very large captures typically have sparse coverage of
               the scene, presenting many challenges to the original 3D
               Gaussian splatting training method; we adapt and regularize
               training to account for these issues. We present a complete
               solution, that enables real-time rendering of very large
               scenes and can adapt to available resources thanks to our
               LOD method. We show results for captured scenes with up to
               tens of thousands of images with a simple and affordable
               rig, covering trajectories of up to several kilometers and
               lasting up to one hour.",
  month =      jul,
  journal =    "ACM Transactions on Graphics",
  volume =     "43",
  number =     "4",
  articleno =  "62",
  issn =       "1557-7368",
  doi =        "10.1145/3658160",
  pages =      "15",
  publisher =  "ASSOC COMPUTING MACHINERY",
  pages =      "1--15",
  keywords =   "real-time rendering, 3d gaussian splatting, level-of-detail,
               Large Scenes",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2024/kerbl-2024-ah3/",
}