Efficient and Practical Audio-Visual Rendering for Games using Crossmodal Perception

David Grelaud, Nicolas Bonneel, Michael Wimmer, Manuel Asselot, George Drettakis
Efficient and Practical Audio-Visual Rendering for Games using Crossmodal Perception
In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2009, pages 177-182. February 2009.
[ Preprint]
Content:

Information

Abstract

Interactive applications such as computer games, are inherently audio visual, requiring high-quality rendering of complex 3D audio soundscapes and graphics environments. A frequent source of audio events is impact sounds, typically generated with physics engines. In this paper, we first present an optimization allowing efficient usage of impact sounds in a unified audio rendering pipeline, also including prerecorded sounds. We also exploit a recent result on audio-visual crossmodal perception to introduce a new level-of-detail selection algorithm, which jointly chooses the quality level of audio and graphics rendering. We have integrated these two techniques as a comprehensive crossmodal audio-visual rendering pipeline in a home-grown game engine, thus demonstrating the potential utility of our approach.

Additional Files and Images

Additional files:
Preprint
Preprint




BibTeX

Download BibTeX-Entry
@inproceedings{GRELAUD-2009-EPA,
  title =      "Efficient and Practical Audio-Visual Rendering for Games
               using Crossmodal Perception",
  author =     "David Grelaud and Nicolas Bonneel and Michael Wimmer and
               Manuel Asselot and George Drettakis",
  year =       "2009",
  abstract =   "Interactive applications such as computer games, are
               inherently audio visual, requiring high-quality rendering of
               complex 3D audio soundscapes and graphics environments. A
               frequent source of audio events is impact sounds, typically
               generated with physics engines. In this paper, we first
               present an optimization allowing efficient usage of impact
               sounds in a unified audio rendering pipeline, also including
               prerecorded sounds. We also exploit a recent result on
               audio-visual crossmodal perception to introduce a new
               level-of-detail selection algorithm, which jointly chooses
               the quality level of audio and graphics rendering. We have
               integrated these two techniques as a comprehensive
               crossmodal audio-visual rendering pipeline in a home-grown
               game engine, thus demonstrating the potential utility of our
               approach.",
  pages =      "177--182",
  month =      feb,
  address =    "New York, NY, USA",
  booktitle =  "Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D
               Graphics and Games 2009",
  isbn =       "978-1-60558-429-4",
  publisher =  "ACM",
  location =   "Boston, Massachusetts",
  keywords =   "audio-visual rendering, crossmodal perception",
  URL =        "http://www.cg.tuwien.ac.at/research/publications/2009/GRELAUD-2009-EPA/",
}