The vOICe at SIGGRAPH 98

The vOICe auditory display technology was outlined and demonstrated at SIGGRAPH 98 (Orange County Convention Center, Orlando, Florida) in an invited panel presentation.

The panel session was titled  "Listen Up! Real-Time Auditory Interfaces for the Real World" and involved a demo of The vOICe for Windows (v1.11) during the panel presentation on July 22, 1998, at 2:15-4 pm. Panel participants: Maribeth Back and Elizabeth Mynatt from Xerox PARC, Robin Bargar from the University of Illinois at Urbana-Champaign / NCSA, Peter B.L. Meijer from Philips Research Laboratories [update: currently no longer working at Philips] and Perry R. Cook from Princeton University. Further demonstrations of The vOICe for Windows were subsequently given at the CAL (Creative Applications Lab) on July 22, 4:15-6 pm, and July 23, 10-12 am.

Panel abstract as published in  Conference Abstracts and Applications, ACM SIGGRAPH 98, 1998 (ISBN 1-58113-046-5), p. 184:

Cross-Modal Sensory Streams (Peter B.L. Meijer)

The rapidly increasing computational power of multimedia PC's is beginning to enable real-time transformations of complex (real-life) sensory streams in software. An exciting challenge is to find out if real-time cross-modal sensory mappings could help in dealing with sensory disabilities, e.g., to feel or hear images if one is blind, to feel or see sounds if one is deaf, or to feel sounds and/or images if one is deaf-blind. This presentation focusses on developing general and affordable auditory displays for the blind, using portable equipment (wearable computing). A demonstration of real-time video sonification will be included, based on an approach named ``The vOICe,'' now running on a regular PC with a webcam.

In using cross-modal sensory mappings, there are inevitable trade-offs between information preservation (both space and time resolution), aesthetic acceptability, and limitations in human perceptual capabilities. The technology for affordable cross-modal mappings is (almost) there, but little understanding currently exists about what mappings and mapping parameters would be best under what circumstances, or what the actual added long-term value of any given mapping would be to the disabled user. Cooperation between engineers, neuroscientists and psychologists to further evaluate the options would be a logical next step. A key issue is that learning to exploit new information-rich auditory displays may require a major training effort, while one does not know in advance if the resulting human performance level would indeed justify that effort.

Slides used at  SIGGRAPH in Orlando, Florida, USA, July 19-24, 1998.
Available as Microsoft PowerPoint file  siggraph98.ppt (700 K).

01   02 

03   04 

05   06 

07   08 

09   10 

11 

See also the web pages on the invited lecture at the VSPA 2001 conference on Consciousness (Amsterdam, The Netherlands), the invited presentation at the Nordic Interactive Conference NIC2001 (Copenhagen, Denmark), and the invited presentation at the Tucson 2002 conference on consciousness (Tucson, Arizona, USA).