Reciprocal Shading for Mixed Reality

The aim of the RESHADE project is to simulate the mutual influence between real and virtual objects in mixed reality applications. Virtual objects in such applications appear disturbingly artificial because rendering completely ignores the real environment. But from the term mixed reality one would expect that virtual and real object harmoniously blend into one visual perception and cannot be distinguished easily.

The ambitious goal of this project is to provide users with a perfect illusion, so that they cannot perceive a difference between virtual and real objects. That means virtual objects are responsive to changes in the real environment in real time. They have to be rendered in a believable realistic way so that they smoothly blend with reality. It is also important to simulate the visual effects of virtual objects on real ones.

Besides for realistic rendering this technique can also be used for innovative ways of interaction and dynamic behaviour of virtual objects, which are key aspects of mixed reality.

Research focuses on real time rendering algorithms for mixed reality applications that consider the lighting and geometry of the real environment. Special cameras will continuously capture reality. The real geometry will also be used to simulate effects of virtual objects on real ones in real time. Shading of real objects will be corrected by digital composition.

Research results will be repeatedly evaluated in the course of the project within three different application scenarios. Thereby human perception is the sole key factor for assessment of the quality of the solution.


  • FFG 820916

Research Areas

Research Area
In this area, we concentrate on algorithms that synthesize images to depict 3D models or scenes, often by simulating or approximating the physics of light.


14 Publications found:
Image Bib Reference Publication Type
Martin Knecht
Reciprocal Shading for Mixed Reality
Supervisor: Michael Wimmer
Duration: August 2009 - December 2013
Martina Rasch
HDR Image Acquisition for Augmented Reality
Bachelor Thesis
Martin Knecht, Christoph Traxler, Christoph Winklhofer, Michael Wimmer
Reflective and Refractive Objects for Mixed Reality
IEEE Transactions on Visualization and Computer Graphics (Proceedings of IEEE VR 2013), 19(4):576-582, March 2013. [Slides]
Journal Paper with Conference Talk
Klemens Jahrmann
Kinect Fusion - Reconstruction
Bachelor Thesis
Florian Laager
Camera Artifacts in Mixed Reality
Student Project
Christoph Winklhofer
Reflections, Refractions and Caustics in a Mixed-Reality Environment
Master Thesis
Adam Celarek
Merging Ray Tracing and Rasterization in Mixed Reality
Bachelor Thesis
Martin Knecht, Christoph Traxler, Oliver Mattausch, Michael Wimmer
Reciprocal Shading for Mixed Reality
Computers & Graphics, 36(7):846-856, November 2012. [draft]
Journal Paper (without talk)
Stefan Spelitz
Color Distribution Transfer For Mixed-Reality Applications
Bachelor Thesis
Martin Knecht, Georg Tanzmeister, Christoph Traxler, Michael Wimmer
Interactive BRDF Estimation for Mixed-Reality Applications
Journal of WSCG, 20(1):47-56, June 2012. [paper]
Journal Paper with Conference Talk
Martin Knecht, Christoph Traxler, Werner Purgathofer, Michael Wimmer
Adaptive Camera-Based Color Mapping For Mixed-Reality Applications
In Proceedings of the 2011 IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2011), pages 165-168. October 2011.
[Preprint] [Video]
Conference Paper
Patrick Kühtreiber, Martin Knecht, Christoph Traxler
BRDF approximation and estimation for Augmented Reality
In 15th International Conference on System Theory, Control and Computing, pages 318-324. October 2011.
Conference Paper
Martin Knecht, Andreas Dünser, Christoph Traxler, Michael Wimmer, Raphael Grasset
A Framework For Perceptual Studies In Photorealistic Augmented Reality
In Proceedings of the 3rd IEEE VR 2011 Workshop on Perceptual Illusions in Virtual Environments, pages 27-32. March 2011.
Conference Paper
Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer
Differential Instant Radiosity for Mixed Reality
In Proceedings of the 2010 IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2010), pages 99-107. October 2010.
Conference Paper
Download list as Bibtex, HTML (Advanced, Expert), JSON (with referenced objects), CSV