Interactive 3D Reconstruction and BRDF Estimation for Mixed Reality Environments | TU Wien

Information

Publication Type: Master Thesis
Workgroup(s)/Project(s):
- Rendering and Modeling
Date: October 2011
Date (Start): 8. April 2011
First Supervisor:
- Martin Knecht
- Michael Wimmer

Abstract

In mixed reality environments virtual content typically looks very artificial. One reason for that is because there is no consistent shading between the virtual and the real objects. Two examples are shadows and indirect illumination between the artificial and the real scene elements, which require to have information about the real world’s geometry and its materials respectively. In mixed reality interaction with real objects is a key feature. Integrating consistent shading in such a system means that the system at all times needs an up-to-date model of the scene’s geometry, its lighting and its material characteristics. The information is usually obtained as a manual pre-processing step, which is a tedious, time-consuming task and has to be re-done whenever a scene element that is not tracked changes. This poses strong limits to the widespread use of such a technique and one would like to have it done automatically. However, the automatic estimation of material characteristics of real objects using color images has always been an offline task in the literature having processing times from around 30 minutes up to several hours. In this work an interactive BRDF estimation technique is proposed, which uses the parallel power of current GPUs speeding up the running time to under half a second. One reason for the speed-up was a novel GPU K-Means implementation using MIP maps to calculate the new cluster centers on the GPU, which is often done on the CPU. The 3D geometry is also reconstructed in our technique since it is needed for indirect illumination and occlusion. We use the Microsoft Kinect sensor to acquire both, the geometry and the color images and capture the lighting environment using a fish-eye lens camera. With the algorithm presented in this thesis we have shown that real-time results are possible opening up its use in mixed reality systems in order to improve the appearance of virtual content.

Additional Files and Images

Poster

Thesis

Weblinks

No further information available.

BibTeX

@mastersthesis{Tanzmeister_2011_I3D,
  title =      "Interactive 3D Reconstruction and BRDF Estimation for Mixed
               Reality Environments",
  author =     "Georg Tanzmeister",
  year =       "2011",
  abstract =   "In mixed reality environments virtual content typically
               looks very artificial. One reason for that is because there
               is no consistent shading between the virtual and the real
               objects. Two examples are shadows and indirect illumination
               between the artificial and the real scene elements, which
               require to have information about the real world’s
               geometry and its materials respectively. In mixed reality
               interaction with real objects is a key feature. Integrating
               consistent shading in such a system means that the system at
               all times needs an up-to-date model of the scene’s
               geometry, its lighting and its material characteristics. The
               information is usually obtained as a manual pre-processing
               step, which is a tedious, time-consuming task and has to be
               re-done whenever a scene element that is not tracked
               changes. This poses strong limits to the widespread use of
               such a technique and one would like to have it done
               automatically. However, the automatic estimation of material
               characteristics of real objects using color images has
               always been an offline task in the literature having
               processing times from around 30 minutes up to several hours.
               In this work an interactive BRDF estimation technique is
               proposed, which uses the parallel power of current GPUs
               speeding up the running time to under half a second. One
               reason for the speed-up was a novel GPU K-Means
               implementation using MIP maps to calculate the new cluster
               centers on the GPU, which is often done on the CPU. The 3D
               geometry is also reconstructed in our technique since it is
               needed for indirect illumination and occlusion. We use the
               Microsoft Kinect sensor to acquire both, the geometry and
               the color images and capture the lighting environment using
               a fish-eye lens camera. With the algorithm presented in this
               thesis we have shown that real-time results are possible
               opening up its use in mixed reality systems in order to
               improve the appearance of virtual content.",
  month =      oct,
  address =    "Favoritenstrasse 9-11/E193-02, A-1040 Vienna, Austria",
  school =     "Institute of Computer Graphics and Algorithms, Vienna
               University of Technology ",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2011/Tanzmeister_2011_I3D/",
}