Amir Amedi ¹, Felix Bermpohl ¹, Joan Camprodon ¹, Sharon Fox ¹, Lotfi Merabet ¹, Peter Meijer ² and Alvaro Pascual-Leone ¹.

¹ Center for Non-Invasive Magnetic Brain Stimulation, Dept. of Neurology, BIDMC, Harvard Medical School, MA02215, USA.
² Philips Research, Eindhoven, The Netherlands. [Update: currently no longer working at Philips.]

Abstract

Restoration of sight in a blind person imposes great clinical and scientific challenges. Despite intensive efforts, restoration of truly functional vision using neuroprostheses has not been achieved. We suggest here that a major reason for this failure is that the brain in the blind undergoes profound plastic changes and we do not know enough about how to communicate with this altered cortex to generate meaningful visual perception. We propose that sensory substitution devices (SSD) can play a major role in two fronts: 1. 'Guide' the visual cortex to 'read' and interpret the visual information arriving from a retinal prosthesis in a blind subject 2. To be useful in daily life for blind people to perceive and recognize visual input, transformed to their intact senses. Visual-to-tactile and visual-to-auditory sensory substitution devices (SSD) can be useful for blind individuals to perceive visual scenes. In SSD, visual images are captured by a camera and then transformed into tactile stimulation or into sound. The brain is able to learn to use this information, but the neural mechanisms responsible for this process have not been elucidated. We report here findings of behavioral and fMRI results in two blind (one congenitally blind and one late blind), expert users of a visual-to-auditory SSD called The vOICe. This system encodes the different aspects of a visual scene (brightness and spatial location) using auditory information ('soundscapes') based upon 3 simple rules: 1) the vertical axis is represented by frequency; 2) the horizontal axis is represented by time and stereo panning; and 3) brightness is encoded by loudness (Meijer, 1992). Two experiments were conducted, one which studied shape versus location processing of visual geometrical shapes transformed into The vOICe format. In the second experiment we compared object recognition using The vOICe with auditory and tactile object recognition. The results suggest that a right hemisphere lateralized network of areas participates in processing The vOICe information. This includes posterior occipital areas, the superior-temporal sulcus, the occipito-temporal cortex and occipito-parietal cortex. Differential activation of ventral V1/V2 is associated with shape versus location decisions using The vOICe. We will further present the difference between tactile and soundscape-based object recognition as well as difference between the congenitally and late blind and suggest a general framework on how to use this result in visual restoration efforts.

Poster

The poster is available as a  PDF file (760 KB).