Streamed Ray-Tracing in Mobile Augmented Reality

Student Project
Master Thesis
1

Description

Ray-tracing algorithm can provide photo-realistic rendering quality of virtual objects in augmented reality setup. However, its high computational cost prohibits its use on mobile devices. A suitable solution to this problem might be to render the image on a high-end PC and stream it to the target device. In this thesis the focus will be on the Microsoft HoloLens 2 device and the ray-traced images will be streamed in real time via Unreal HoloLens Streaming. UnrealEngine supports GPU ray tracing on RTX graphics cards. This ray-tracing will be adopted to AR rendering and streamed to the HoloLens device. Finally, the created rendering setup will be evaluated in terms of speed and visual quality.

Tasks

This topic can be done as a Practicum or as a Master thesis. In case of practicum the main focus will be on the integration of real-time ray tracing with Hololens Streaming and its deployment to HoloLens device. In case of master thesis this streaming setup will be used to implement advanced rendering techniques for various visual effects. These implemented techniques might include one of the following: Full path tracing with de-noising, world reflection with an additional fish-eye lens camera, photon mapping, image-based lighting, depth of field, etc.

Requirements

  • Knowledge of English language (source code comments and final report should be in English)
  • Knowledge of C++
  • Knowledge of rendering algorithms
  • Previous experience with shader programming or cuda programming is advantageous.

Environment

The project should be implemented in Unreal Engine and it will be deployed on HoloLens 2 device. Eventually, another real-time ray-tracing framework might be an option.

Responsible

For more information please contact Peter Kán, Hannes Kaufmann.