Realistic Rendering in Mobile Augmented Reality

Information

  • Publication Type: Master Thesis
  • Workgroup(s)/Project(s): not specified
  • Date: October 2016
  • Second Supervisor: Peter Kán
  • Diploma Examination: 15. November 2016
  • Open Access: no
  • First Supervisor: Hannes Kaufmann
  • Keywords: Augmented Reality, Mobile, Vuforia, Irradiance Mapping, Reflection Mapping, Spherical Harmonics

Abstract

Augmented Reality (AR) applications combine a view of a physical, real-world environment with computer-generated objects and effects in real-time. Depending on the application, it is desirable to maximize the visual coherence of the virtual objects compared to the real-world image. To achieve this goal, virtual objects have to be rendered as realistically as possible. This thesis presents an image-based lighting (IBL) technique for realistic rendering of virtual objects on mobile devices which uses lighting information from the real-world environment. In the first step, the presented technique uses a mobile device’s camera and motion sensors to capture an omni-directional image of the surrounding in high dynamic range (HDR) and stores it in an environment map. In the second step, the captured environment map is prepared for rendering with different materials by calculating a set of maps. During rendering, the most suitable of these maps are selected for each material and used for shading a virtual object with the specific material. The map which contains diffuse illumination information is called irradiance map, and the maps which contain glossy or specular illumination information are called reflection maps. The calculation of the maps corresponds to a weighted convolution. The weighting is determined by a reflection model which takes the correct amount of incident lighting from all directions into account. How these calculations can be performed efficiently on mobile devices is the main focus of this thesis. Multiple approaches to perform the calculations are described. Their properties, results, strengths and weaknesses are analyzed and optimizations are proposed. We describe three different approaches for the calculation of irradiance and reflection maps in this thesis: the accurate calculation, a MIP-mapping based approximation method, and calculation via spherical harmonics (SH) frequency space. We provide detailed implementation instructions, analyses, and discussions for each of these approaches with regard to the properties and limitations of mobile devices. Furthermore, we describe how the calculated maps can be used with IBL rendering and be combined with established rendering techniques to achieve a high degree of visual coherence of virtual objects in AR scenes. The main novelty of this thesis is its focus on the capabilities of mobile devices. We describe how to do all steps on a single commodity mobile device: From capturing the environment at a certain point in space, to calculating the irradiance and reflection maps, and finally rendering virtual objects using the calculated maps in an AR scene.

Additional Files and Images

Additional images and videos

Additional files

Weblinks

No further information available.

BibTeX

@mastersthesis{unterguggenberger-2016-realmar,
  title =      "Realistic Rendering in Mobile Augmented Reality",
  author =     "Johannes Unterguggenberger",
  year =       "2016",
  abstract =   "Augmented Reality (AR) applications combine a view of a
               physical, real-world environment with computer-generated
               objects and effects in real-time. Depending on the
               application, it is desirable to maximize the visual
               coherence of the virtual objects compared to the real-world
               image. To achieve this goal, virtual objects have to be
               rendered as realistically as possible. This thesis presents
               an image-based lighting (IBL) technique for realistic
               rendering of virtual objects on mobile devices which uses
               lighting information from the real-world environment. In the
               first step, the presented technique uses a mobile device’s
               camera and motion sensors to capture an omni-directional
               image of the surrounding in high dynamic range (HDR) and
               stores it in an environment map. In the second step, the
               captured environment map is prepared for rendering with
               different materials by calculating a set of maps. During
               rendering, the most suitable of these maps are selected for
               each material and used for shading a virtual object with the
               specific material. The map which contains diffuse
               illumination information is called irradiance map, and the
               maps which contain glossy or specular illumination
               information are called reflection maps. The calculation of
               the maps corresponds to a weighted convolution. The
               weighting is determined by a reflection model which takes
               the correct amount of incident lighting from all directions
               into account. How these calculations can be performed
               efficiently on mobile devices is the main focus of this
               thesis. Multiple approaches to perform the calculations are
               described. Their properties, results, strengths and
               weaknesses are analyzed and optimizations are proposed. We
               describe three different approaches for the calculation of
               irradiance and reflection maps in this thesis: the accurate
               calculation, a MIP-mapping based approximation method, and
               calculation via spherical harmonics (SH) frequency space. We
               provide detailed implementation instructions, analyses, and
               discussions for each of these approaches with regard to the
               properties and limitations of mobile devices. Furthermore,
               we describe how the calculated maps can be used with IBL
               rendering and be combined with established rendering
               techniques to achieve a high degree of visual coherence of
               virtual objects in AR scenes. The main novelty of this
               thesis is its focus on the capabilities of mobile devices.
               We describe how to do all steps on a single commodity mobile
               device: From capturing the environment at a certain point in
               space, to calculating the irradiance and reflection maps,
               and finally rendering virtual objects using the calculated
               maps in an AR scene.",
  month =      oct,
  address =    "Favoritenstrasse 9-11/E193-02, A-1040 Vienna, Austria",
  school =     "Institute of Computer Graphics and Algorithms, Vienna
               University of Technology ",
  keywords =   "Augmented Reality, Mobile, Vuforia, Irradiance Mapping,
               Reflection Mapping, Spherical Harmonics",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2016/unterguggenberger-2016-realmar/",
}