Realistic Humanoid Agent in Virtual Reality

Master Thesis
1

Description

The main goal of this thesis will be to create a realistic virtual agent (digital character) which will be capable of voice communication with a user. The realistic appearance will be achieved via Unreal MetaHuman creator. The character models from this tool are prepared for full-body animation, facial emotions and lip-sync. The lip-sync will be used to synchronize the animation with the synthetized voice. Additionally, NVIDIA Jarvis and Rasa frameworks will be integrated into Unreal Engine to achieve intelligent answers, voice response and natural language processing. The aim is to reach a high realism of the created digital character in terms of visual, communicational and animation aspects.

Tasks

The student will create an Unreal Engine project with realistic MetaHuman character from Unreal MetaHuman Creator. NVIDIA Jarvis and Rasa frameworks will need to be set up for conversation with a user. These frameworks will then be integrated by the student into Unreal Engine and synthesized voice will be connected with the facial animation of the character. Finally, realistic human-body animation will be applied to increase the realism.

Requirements

  • Knowledge of C++ is advantageous
  • Previous experience with game engines or virtual reality are welcome

Environment

The project will be implemented in Unreal Engine

Responsible

For more information please contact Peter Kán, Hannes Kaufmann.