Construction of hierarchical 3d gaussian mixture models

Type: 
PR
Persons: 
1
Workgroup: 

Description

A Gaussian mixture model (GMM) is a weighted sum (linear combination) of several Gaussian functions with independent centres and widths. These are often used to describe arbitrary probability density functions (PDFs). As an example, one could cluster points in 1d, fit a Gaussian to each cluster, find weights and voila, you have an estimate for the PDF of all the points.

1d and 3d GMMs

3d GMMs are very powerful as a geometric representation format:

  • They are continuous (unlike a volumetric grid for instance) and defined for the whole space.
  • It is easy to compute things like integrals, derivatives (gradients) and the Fourier transform.
  • Compared to a raw point cloud or volumetric grid they are memory efficient.

We want to experiment with hierarchical GMM representations of point clouds and deep learning. Therefore an efficient implementation of a GMM construction method is needed. It should be capable of constructing GMMs from hundreds of thousands of points in a short enough time so that it can be executed on the fly during learning.

This PhD thesis looks like a promising candidate, but unfortunately no code is available. However you could use it as guidance. Don't be intimidated by the length of the thesis, only a small part is relevant. Another (smaller) task would be visualisation of the GMM (volume rendering, iso surface rendering etc.) for debugging and parameter tweaking purposes.

All in all I think this can be a compact project, which is close to research and if you are interested in deep learning, it could also lead to a master thesis. The code will be released as an open source project, which you can use in your portfolio :)

Task

  • Implement a hierarchical GMM construction method using C++
  • Port the construction method to the GPU using HIP or CUDA
  • Implement methods to visualise the GMMs

Requirements

  • C++
  • parallel programming
  • OpenGL or Vulcan (for the visualisation)
  • HIP or CUDA experience is a bonus.

Environment

Cross platform, cmake, git, and all from above

Contact

For more information please contact Adam Celarek (celarek@cg.tuwien.ac.at) or Michael Wimmer (wimmer@cg.tuwien.ac.at).