Pictures in Your Mind: Using Interactive Gesture-Controlled Reliefs to Explore Art

Andreas Reichinger, H. G. Carrizosa, J. Wood, S. Schröder, C. Löw, Laura R. Luidolt, Maria Schimkowitsch, Anton Fuhrmann, Stefan Maierhofer, Werner Purgathofer
Pictures in Your Mind: Using Interactive Gesture-Controlled Reliefs to Explore Art
ACM Transactions on Accessible Computing, (2), March 2018.

Information

  • Publication Type: Journal Paper (without talk)
  • Workgroup(s)/Project(s):
  • Date: March 2018
  • Journal: ACM Transactions on Accessible Computing
  • Number: 2

Abstract

Tactile reliefs offer many benefits over the more classic raised line drawings or tactile diagrams, as depth, 3D shape, and surface textures are directly perceivable. Although often created for blind and visually impaired (BVI) people, a wider range of people may benefit from such multimodal material. However, some reliefs are still difficult to understand without proper guidance or accompanying verbal descriptions, hindering autonomous exploration. In this work, we present a gesture-controlled interactive audio guide (IAG) based on recent low-cost depth cameras that can be operated directly with the hands on relief surfaces during tactile exploration. The interactively explorable, location-dependent verbal and captioned descriptions promise rapid tactile accessibility to 2.5D spatial information in a home or education setting, to online resources, or as a kiosk installation at public places. We present a working prototype, discuss design decisions, and present the results of two evaluation studies: the first with 13 BVI test users and the second follow-up study with 14 test users across a wide range of people with differences and difficulties associated with perception, memory, cognition, and communication. The participant-led research method of this latter study prompted new, significant and innovative developments.

Additional Files and Images

No additional files or images.

Weblinks

No further information available.

BibTeX

@article{Reichinger-2018-TAC,
  title =      "Pictures in Your Mind: Using Interactive Gesture-Controlled
               Reliefs to Explore Art",
  author =     "Andreas Reichinger and H. G.  Carrizosa and J. Wood and S.
               Schr\"{o}der and C. L\"{o}w and Laura R. Luidolt and Maria
               Schimkowitsch and Anton Fuhrmann and Stefan Maierhofer and
               Werner Purgathofer",
  year =       "2018",
  abstract =   "Tactile reliefs offer many benefits over the more classic   
                            raised line drawings or tactile diagrams, as
               depth, 3D shape,                 and surface textures are
               directly perceivable. Although often                 created
               for blind and visually impaired (BVI) people, a wider       
                        range of people may benefit from such multimodal
               material.                 However, some reliefs are still
               difficult to understand without                 proper
               guidance or accompanying verbal descriptions, hindering     
                          autonomous exploration.                          
                      In this work, we present a gesture-controlled
               interactive audio                 guide (IAG) based on
               recent low-cost depth cameras that can be                
               operated directly with the hands on relief surfaces during  
                             tactile exploration. The interactively
               explorable,                 location-dependent verbal and
               captioned descriptions promise                 rapid tactile
               accessibility to 2.5D spatial information in a              
                 home or education setting, to online resources, or as a
               kiosk                 installation at public places.        
                                        We present a working prototype,
               discuss design decisions, and                 present the
               results of two evaluation studies: the first with 13        
                       BVI test users and the second follow-up study with
               14 test users                 across a wide range of people
               with differences and difficulties                 associated
               with perception, memory, cognition, and                
               communication. The participant-led research method of this  
                             latter study prompted new, significant and
               innovative                 developments. ",
  month =      mar,
  journal =    "ACM Transactions on Accessible Computing",
  number =     "2",
  URL =        "/research/publications/2018/Reichinger-2018-TAC/",
}