Amir Amedi ¹, Lotfi Merabet ¹, Felix Bermpohl ¹, Joan Camprodon ¹, Sharon Fox ¹, and Alvaro Pascual-Leone ¹.

¹ Center for Non-Invasive Magnetic Brain Stimulation, Dept. of Neurology, BIDMC, Harvard Medical School, MA02215, USA.

Objective

Restoration of sight in a blind person imposes great clinical and scientific challenges. Despite intensive efforts, restoration of truly functional vision using neuroprostheses has not been achieved. We suggest here that a major reason for this failure is that the brain in the blind undergoes profound plastic changes and we do not know enough about how to communicate with this altered cortex to generate meaningful visual perception. We propose that sensory substitution devices (SSD) can play a major role in two fronts:

1. 'Guide' the visual cortex to 'read' and interpret the visual information arriving from a retinal prosthesis in a blind subject.
2. To be useful in daily life for blind people to perceive and recognize visual input, transformed to their intact senses.

Visual-to-tactile and visual-to-auditory sensory substitution devices can be useful for blind individuals to perceive visual scenes.In SSD, visual images are captured by a camera and then transformed into tactile stimulation or into sound. The brain is able to learn to use this information, but the neural mechanisms responsible for this process have not been elucidated.

Design/Methods

We report here findings of behavioral and fMRI results in two blind (one congenital and one late blind), expert users of a visual-to-auditory SSD called The vOICe. This system encodes the different aspects of a visual scene (brightness and spatial location) using auditory information ('sound-scapes') based upon 3 simple rules: 1) the vertical axis is represented by frequency; 2) the horizontal axis is represented by time and stereo panning; and 3) brightness is encoded by loudness (Meijer, 1992). Two experiments were conducted, one which studied shape versus location processing of visual geometrical shapes transformed into the vOICe format. In the second experiment, we compared object recognition using The vOICe with auditory and tactile object recognition.

Results

The results suggest that a right hemisphere lateralized network of areas participates in processing the vOICe information. This includes posterior occipital areas, the superior-temporal sulcus, the occipito-temporal cortex and occipito-parietal cortex. Differential activation of ventral V1/V2 is associated with shape versus location decisions using The vOICe.

Conclusions

To conclude, a constellation of areas including early visual cortex is involved in the visual-to-auditory transformation of visual scenes in blind subjects. The specific recruitment of the ventral visual stream in shape processing suggests that this SSD might be suitable to help in the adjustment to visual input arriving from implanted neuronal prostheses. We will further present the difference between tactile and soundscape-based object recognition as well as differences between congenital and late blind individuals and suggest a general framework on how to use these results in visual restoration efforts.