Differential Instant Radiosity for Mixed Reality

Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer
Differential Instant Radiosity for Mixed Reality
In Proceedings of the 2010 IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2010), pages 99-107. October 2010.
[paper]

Information

Abstract

In this paper we present a novel plausible realistic rendering method for mixed reality systems, which is useful for many real life application scenarios, like architecture, product visualization or edutainment. To allow virtual objects to seamlessly blend into the real environment, the real lighting conditions and the mutual illumination effects between real and virtual objects must be considered, while maintaining interactive frame rates (20-30fps). The most important such effects are indirect illumination and shadows cast between real and virtual objects.

Our approach combines Instant Radiosity and Differential Rendering. In contrast to some previous solutions, we only need to render the scene once in order to find the mutual effects of virtual and real scenes. The dynamic real illumination is derived from the image stream of a fish-eye lens camera. We describe a new method to assign virtual point lights to multiple primary light sources, which can be real or virtual. We use imperfect shadow maps for calculating illumination from virtual point lights and have significantly improved their accuracy by taking the surface normal of a shadow caster into account. Temporal coherence is exploited to reduce flickering artifacts. Our results show that the presented method highly improves the illusion in mixed reality applications and significantly diminishes the artificial look of virtual objects superimposed onto real scenes.

Additional Files and Images

Additional images and videos

image1: Virtual objects shadow real box image1: Virtual objects shadow real box
image2: Color bleeding from virtual on real object image2: Color bleeding from virtual on real object
image3: Color bleeding from real on virtual object image3: Color bleeding from real on virtual object
Video: Video Results Video: Video Results

Additional files

paper: Differential Instant Radiosity for Mixed Reality paper: Differential Instant Radiosity for Mixed Reality

Weblinks

No further information available.

BibTeX

@inproceedings{knecht_martin_2010_DIR,
  title =      "Differential Instant Radiosity for Mixed Reality",
  author =     "Martin Knecht and Christoph Traxler and Oliver Mattausch and
               Werner Purgathofer and Michael Wimmer",
  year =       "2010",
  abstract =   "In this paper we present a novel plausible realistic
               rendering method for mixed reality systems, which is useful
               for many real life application scenarios, like architecture,
               product visualization or edutainment. To allow virtual
               objects to seamlessly blend into the real environment, the
               real lighting conditions and the mutual illumination effects
               between real and virtual objects must be considered, while
               maintaining interactive frame rates (20-30fps). The most
               important such effects are indirect illumination and shadows
               cast between real and virtual objects.  Our approach
               combines Instant Radiosity and Differential Rendering. In
               contrast to some previous solutions, we only need to render
               the scene once in order to find the mutual effects of
               virtual and real scenes. The dynamic real illumination is
               derived from the image stream of a fish-eye lens camera. We
               describe a new method to assign virtual point lights to
               multiple primary light sources, which can be real or
               virtual. We use imperfect shadow maps for calculating
               illumination from virtual point lights and have
               significantly improved their accuracy by taking the surface
               normal of a shadow caster into account. Temporal coherence
               is exploited to reduce flickering artifacts. Our results
               show that the presented method highly improves the illusion
               in mixed reality applications and significantly diminishes
               the artificial look of virtual objects superimposed onto
               real scenes.",
  month =      oct,
  booktitle =  "Proceedings of the 2010 IEEE International Symposium on
               Mixed and Augmented Reality (ISMAR 2010)",
  location =   "Seoul",
  note =       "Best Paper Award!",
  pages =      "99--107",
  keywords =   "Instant Radiosity, Differential Rendering, Real-time Global
               Illumination, Mixed Reality",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2010/knecht_martin_2010_DIR/",
}