Mohammadreza Mirzaei, Peter Kán, Hannes KaufmannORCID iD
Head Up Visualization of Spatial Sound Sources in Virtual Reality for Deaf and Hard-of-Hearing People
In IEEE Virtual Reality and 3D User Interfaces (VR), pages 582-587. 2021.

Information

  • Publication Type: Conference Paper
  • Workgroup(s)/Project(s): not specified
  • Date: 2021
  • Publisher: IEEE Computer Society
  • Event: IEEE VR 2021
  • DOI: 10.1109/VR50410.2021.00083
  • Booktitle: IEEE Virtual Reality and 3D User Interfaces (VR)
  • Pages: 6
  • Pages: 582 – 587
  • Keywords: Virtual Reality, Haptic, Vision, Sound Localization, Deaf, Hard-of-Hearing

Abstract

This paper presents a novel method for the visualization of 3D spatial sounds in Virtual Reality (VR) for Deaf and Hard-of-Hearing (DHH) people. Our method enhances traditional VR devices with additional haptic and visual feedback, which aids spatial sound localization. The proposed system automatically analyses 3D sound from VR application, and it indicates the direction of sound sources to a user by two Vibro-motors and two Light-Emitting Diodes (LEDs). The benefit of automatic sound analysis is that our method can be used in any VR application without modifying the application itself. We evaluated the proposed method for 3D spatial sound visualization in a user study. Additionally, the conducted user study investigated which condition (corresponding to different senses) leads to faster performance in 3D sound localization task. For this purpose, we compared three conditions: haptic feedback only, LED feedback only, combined haptic and LED feedback. Our study results suggest that DHH participants could complete sound-related VR tasks significantly faster using LED and haptic+LED conditions in comparison to only haptic feedback. The presented method for spatial sound visualization can be directly used to enhance VR applications for use by DHH persons, and the results of our user study can serve as guidelines for the future design of accessible VR systems.

Additional Files and Images

Additional images and videos

head-up: Head up display extended with a LEDs (on both sides) for indicating direction of incoming sound head-up: Head up display extended with a LEDs (on both sides) for indicating direction of incoming sound

Additional files

Weblinks

BibTeX

@inproceedings{Mirzaei_Mohammad-2021-HeadUp,
  title =      "Head Up Visualization of Spatial Sound Sources in Virtual
               Reality for Deaf and Hard-of-Hearing People",
  author =     "Mohammadreza Mirzaei and Peter K\'{a}n and Hannes Kaufmann",
  year =       "2021",
  abstract =   "This paper presents a novel method for the visualization of
               3D spatial sounds in Virtual Reality (VR) for Deaf and
               Hard-of-Hearing (DHH) people. Our method enhances
               traditional VR devices with additional haptic and visual
               feedback, which aids spatial sound localization. The
               proposed system automatically analyses 3D sound from VR
               application, and it indicates the direction of sound sources
               to a user by two Vibro-motors and two Light-Emitting Diodes
               (LEDs). The benefit of automatic sound analysis is that our
               method can be used in any VR application without modifying
               the application itself. We evaluated the proposed method for
               3D spatial sound visualization in a user study.
               Additionally, the conducted user study investigated which
               condition (corresponding to different senses) leads to
               faster performance in 3D sound localization task. For this
               purpose, we compared three conditions: haptic feedback only,
               LED feedback only, combined haptic and LED feedback. Our
               study results suggest that DHH participants could complete
               sound-related VR tasks significantly faster using LED and
               haptic+LED conditions in comparison to only haptic feedback.
               The presented method for spatial sound visualization can be
               directly used to enhance VR applications for use by DHH
               persons, and the results of our user study can serve as
               guidelines for the future design of accessible VR systems.",
  publisher =  "IEEE Computer Society",
  event =      "IEEE VR 2021",
  doi =        "10.1109/VR50410.2021.00083",
  booktitle =  "IEEE Virtual Reality and 3D User Interfaces (VR)",
  pages =      "6",
  pages =      "582--587",
  keywords =   "Virtual Reality, Haptic, Vision, Sound Localization, Deaf,
               Hard-of-Hearing",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2021/Mirzaei_Mohammad-2021-HeadUp/",
}