Information
- Publication Type: Master Thesis
- Workgroup(s)/Project(s): not specified
- Date: October 2016
- Date (Start): July 2014
- Date (End): October 2016
- Second Supervisor: Peter Kán
- Diploma Examination: 15. November 2016
- Open Access: no
- First Supervisor: Hannes Kaufmann
- Keywords: Augmented Reality, Mobile, Vuforia, Irradiance Mapping, Reflection Mapping, Spherical Harmonics
Abstract
Augmented Reality (AR) applications combine a view of a physical, real-world environment
with computer-generated objects and effects in real-time. Depending on the application,
it is desirable to maximize the visual coherence of the virtual objects compared to the
real-world image. To achieve this goal, virtual objects have to be rendered as realistically
as possible. This thesis presents an image-based lighting (IBL) technique for realistic
rendering of virtual objects on mobile devices which uses lighting information from the
real-world environment.
In the first step, the presented technique uses a mobile device’s camera and motion sensors
to capture an omni-directional image of the surrounding in high dynamic range (HDR)
and stores it in an environment map. In the second step, the captured environment map
is prepared for rendering with different materials by calculating a set of maps. During
rendering, the most suitable of these maps are selected for each material and used for
shading a virtual object with the specific material. The map which contains diffuse
illumination information is called irradiance map, and the maps which contain glossy or
specular illumination information are called reflection maps. The calculation of the maps
corresponds to a weighted convolution. The weighting is determined by a reflection model
which takes the correct amount of incident lighting from all directions into account. How
these calculations can be performed efficiently on mobile devices is the main focus of this
thesis. Multiple approaches to perform the calculations are described. Their properties,
results, strengths and weaknesses are analyzed and optimizations are proposed.
We describe three different approaches for the calculation of irradiance and reflection maps
in this thesis: the accurate calculation, a MIP-mapping based approximation method,
and calculation via spherical harmonics (SH) frequency space. We provide detailed
implementation instructions, analyses, and discussions for each of these approaches with
regard to the properties and limitations of mobile devices. Furthermore, we describe how
the calculated maps can be used with IBL rendering and be combined with established
rendering techniques to achieve a high degree of visual coherence of virtual objects in
AR scenes.
The main novelty of this thesis is its focus on the capabilities of mobile devices. We
describe how to do all steps on a single commodity mobile device: From capturing the
environment at a certain point in space, to calculating the irradiance and reflection maps,
and finally rendering virtual objects using the calculated maps in an AR scene.
Additional Files and Images
Additional images and videos
Additional files
Weblinks
No further information available.
BibTeX
@mastersthesis{unterguggenberger-2016-realmar,
title = "Realistic Rendering in Mobile Augmented Reality",
author = "Johannes Unterguggenberger",
year = "2016",
abstract = "Augmented Reality (AR) applications combine a view of a
physical, real-world environment with computer-generated
objects and effects in real-time. Depending on the
application, it is desirable to maximize the visual
coherence of the virtual objects compared to the real-world
image. To achieve this goal, virtual objects have to be
rendered as realistically as possible. This thesis presents
an image-based lighting (IBL) technique for realistic
rendering of virtual objects on mobile devices which uses
lighting information from the real-world environment. In the
first step, the presented technique uses a mobile device’s
camera and motion sensors to capture an omni-directional
image of the surrounding in high dynamic range (HDR) and
stores it in an environment map. In the second step, the
captured environment map is prepared for rendering with
different materials by calculating a set of maps. During
rendering, the most suitable of these maps are selected for
each material and used for shading a virtual object with the
specific material. The map which contains diffuse
illumination information is called irradiance map, and the
maps which contain glossy or specular illumination
information are called reflection maps. The calculation of
the maps corresponds to a weighted convolution. The
weighting is determined by a reflection model which takes
the correct amount of incident lighting from all directions
into account. How these calculations can be performed
efficiently on mobile devices is the main focus of this
thesis. Multiple approaches to perform the calculations are
described. Their properties, results, strengths and
weaknesses are analyzed and optimizations are proposed. We
describe three different approaches for the calculation of
irradiance and reflection maps in this thesis: the accurate
calculation, a MIP-mapping based approximation method, and
calculation via spherical harmonics (SH) frequency space. We
provide detailed implementation instructions, analyses, and
discussions for each of these approaches with regard to the
properties and limitations of mobile devices. Furthermore,
we describe how the calculated maps can be used with IBL
rendering and be combined with established rendering
techniques to achieve a high degree of visual coherence of
virtual objects in AR scenes. The main novelty of this
thesis is its focus on the capabilities of mobile devices.
We describe how to do all steps on a single commodity mobile
device: From capturing the environment at a certain point in
space, to calculating the irradiance and reflection maps,
and finally rendering virtual objects using the calculated
maps in an AR scene.",
month = oct,
address = "Favoritenstrasse 9-11/E193-02, A-1040 Vienna, Austria",
school = "Institute of Computer Graphics and Algorithms, Vienna
University of Technology ",
keywords = "Augmented Reality, Mobile, Vuforia, Irradiance Mapping,
Reflection Mapping, Spherical Harmonics",
URL = "https://www.cg.tuwien.ac.at/research/publications/2016/unterguggenberger-2016-realmar/",
}
poster
thesis
