Bionic Vision for the Blind

In recent years, progress has been made towards development of electronic and optoelectronic retinal implant technology for the blind, sometimes referred to as "retinal prosthesis", "artificial retina" or even "bionic eye". In the long run, there could be the possibility of a brain implant. Now all of these approaches require surgery, and the associated implantable medical devices still count as highly experimental and need to prove their value in for instance ambulatory vision - the ability to move around using only visual information.

Moreover, retinal implants and brain implants will probably not benefit adult early-blind or congenitally blind (blind-from-birth) people much if at all, because their visual cortex has developed differently for lack of visual input. However, the auditory system of early blind people is often very well developed - in fact exceeding the performance of sighted people in a variety of auditory and verbal tasks, so an auditory display could in principle make a better match. This is what The vOICe technology is about. Will this lead to be a "disruptive technology" that displaces other technologies? You decide.

Note: The vOICe technology is intended for educational uses, to stimulate academic and clinical research, to learn about vision and to enable training for visual skills and experiences. Unlike typical retinal implant technologies, it is not intended to treat, cure, or prevent any disease or condition.

The vOICe is being used not only by early-blind people but also by late-blind people. Initially, late-blind users may be trying to mentally (re)construct images from the sounds, i.e., through "mental imagery" and their rational understanding of the image to sound mapping principles, but it is hoped that in the longer run, as users become more "fluent" in this process, it will become more like "seeing" without conscious effort. One underlying assumption here is that the brain is ultimately not interested in the information carrier, here sound, but only in the information content, which is the visual view from a head-mounted camera. (After all, the signals in the optic nerve of a normally sighted person are also "just" neural spiking patterns, not at that point intrinsically tied to a specific physical modality. What you think you "see" is what your brain makes of all those firing patterns.)

Now can we make a comparison against retinal chip implants?

Technically, The vOICe offers a higher image resolution, up to several thousand pixels (e.g., 60 by 60 pixels) than the retinal implants that are currently still under development. Also, as stated before, it does not require any surgery (with the associated risks). In fact, its interface to the brain via sound is wireless. In addition, The vOICe approach makes use of only mature and hence reliable mass-market hardware components that one can buy in most computer shops: PC camera, subnotebook PC and stereo headphones (COTS - Commercial Off the Shelf). This also helps to keep the cost down as compared to more dedicated medical hardware technologies. And yet The vOICe directly builds upon Nature's own nanotechnology, including some of the most advanced and sensitive nanoscale MEMS, namely human hearing, which can register membrane displacements right down to the nanometer scale. All of this is achieved without any of the health or safety concerns about possible hazards with human-made nanoparticles and nanotechnology in general.

The effect of image resolution is further illustrated with the images below, where a photograph of a parked car has been pixelized to 4 by 4, 8 by 8, 12 by 12, 16 by 16, 32 by 32, 64 by 64 and 128 by 128 pixels, respectively. The images here still include shading, while some implants may give little more than on/off signals per pixel or phosphene.

16 pixels (4 × 4)	64 pixels (8 × 8)	144 pixels (12 × 12)	256 pixels (16 × 16)	1024 pixels (32 × 32)	4096 pixels (64 × 64)	16384 pixels (128 × 128)

Visitors may download these images and next import them one by one into The vOICe for Windows via its File menu, or by pressing Control O to get to its file requester. For your convenience, the image set is available for download as a 28K zip file cartest.zip or as an animated GIF image, or you can listen to the  Visual resolution with The vOICe video clip on YouTube. Thus you can try it yourself and listen for any noticeable differences - or lack thereof - as you pick different image resolutions:

Contrary to The vOICe, the hardware technology for retinal implants is still very new and experimental, and is likely to show technical and biocompatibility problems for a long time to come. The insertion of a foreign object in the eye in the form of chronic implantation of a retinal implant may have a number of undesirable side effects, such as retinal microaneurysm formation, damage to retinal capillaries, bleeding and retinal detachment. Can the resolution of retinal implants be increased without exceeding safety limits for current density? Can one electrically stimulate the retina for many years without damaging it beyond repair? It is known that chronic neural stimulation by electrodes can damage or kill neurons, and in fact any over-excitation may lead to nerve cell death. Apart from this, great care must be taken to ensure that microchip implants will not increase the risk of cancer, through foreign body reactions. With The vOICe, typically 100 % of the pixels are fully functional, hence giving a "pixel-perfect" view, and through the use of hearing it can be considered 100 percent biocompatible with the body. Moreover, we know that musicians can - if they are careful about loudness - work with sound on a daily basis for many decades without significant (hearing) damage. On the other hand, a key advantage of retinal implants for late-blinded people could be that they may more readily be able to make (some) sense of their retinal implant view, plus that they immediately regain some light perception, because the view is encoded in much the same way that they grew up with when they still had their normal eyesight: their brain is probably still well-prepared for it, although the field of view with retinal implants will likely remain narrow (much like with severe tunnel vision) for some time to come, due to size constraints with the retinal chip implant contact area. A possibility is to apply a hybrid approach where The vOICe provides wide-field peripheral visual input to compensate for the retinal implants' viewing angle deficiencies and spatial sampling jitter.

Original camera images (left) and spectrographic reconstructions from The vOICe soundscapes (right).

Retinal implants might in the long run outperform The vOICe in resolution and frame rate, because the ultimate resolution and frame rate of The vOICe will in part be limited by the so-called frequency-time uncertainty relation that applies to all sound, but retinal implants would still only be applicable for late-blind people who have specific eye diseases that lead to rod and cone degeneration, such as Retinitis Pigmentosa (RP) or age-related macular degeneration (AMD/MD/ARMD), but that leave the neural connections from the retina to the brain mostly intact and in the proper positions for making contacts with implanted electrodes. Essentially, retinal implants are only designed to take over the role of the photoreceptor cells. People with other causes of blindness, such as diabetic retinopathy, severe optic atrophy, retinal detachment, glaucoma, stroke or trauma, remain unlikely to benefit much if anything from eye surgery for an electronic retinal implant. People affected by retinopathy of prematurity (ROP, leading to retinal scarring and detachment) also seem unlikely to benefit from a retinal implant. The same applies to early-blind people in general, unless they obtained a functional implant at an early age. The vOICe generally seems to offer better chances for early-blind people, because it invokes the auditory cortex and association cortex, and it is even known from recent neuroscience research that auditory input can activate the "visual" cortex of early-blind people ("visual" between quotes here, because this part of the cortex - the occipital lobe - probably takes on different functional roles in early-blind people). A fundamental research question is what happens when the auditory input represents visual content. Preliminary neuroscience findings based on The vOICe were presented at CNS 2005 in New York. To the extent that the auditory regions of the brain can learn to contribute to visual functions themselves rather than "just" relaying the visual information on to the visual areas, The vOICe may even provide forms of vision to those afflicted by cortical blindness (blindness due to damage to the visual cortex, e.g., through trauma or stroke). Neither retinal implants nor implants in the visual cortex would be of any use here.

Nevertheless, the way that visual information is encoded can have a dramatic impact on our (in)ability to make sense of things. An example is the demonstration for the sighted showing a parked car scene as a "3D mountain plot" (brightness here making the third dimension), and although the information content is about the same as in the original view - apart from some occlusion effects, it is very difficult for normally sighted people to "see" anything meaningful here. In other words, a major learning and adaptation task can be expected for differently encoded perceptual input. It is not yet known how far "fluency" in using The vOICe will go, but The vOICe is already in use by both late-blind and early-blind users. Recipients of a retinal implant may also be affected by learning and adaptation issues - although perhaps to a lesser extent - since stimulating a single electrode in contact with the retina does not always yield the percept of a single bright spot. Adaptation issues similar to those reported for cochlear implant recipients may occur. There exist plasticity and retinal remodeling effects due to retinal disease (e.g., photoreceptor degeneration) and lack of visual input to retinal neurons and the brain (sensory deafferentation), possibly limiting the therapeutic options with implants. The vOICe attempts to take advantage of the spontaneous cortical remodeling that leads to activation of occipital cortex ("visual cortex") by sound in blind people.

For more information about various artificial retina projects and clinical testing of implants, you can visit the websites of  Retina Implant AG in Germany (1600-electrode Alpha AMS, Walter Wrobel, Eberhart Zrenner), out of business as of 2019 and no longer online,  Second Sight Medical Products, Inc. (Argus II epiretinal implant, 60 electrodes arranged in 10 × 6 array, Alfred Mann, Robert Greenberg, out of production as of late 2019) the  Boston Retinal Implant Project (Joseph Rizzo, John Wyatt),  Nano Retina, Inc. in Israel (NR600 epiretinal implant with 676 penetrating electrodes, Yaakov Milstain, Yossi Gross, Jim Von Ehr, Bio-Retina),  Pixium Vision (IRIS, subretinal implant with 150 electrodes, Prima, 378 electrodes, out of business as of 2024),  Science Corp. (Science Eye epiretinal implant with unspecified electrode count), and  IntelliMicro Medical Co., Ltd. in China (IMIE Intelligent Visual Implant, epiretinal implant implant with 256 electrodes). The European Vision Institute European Economic Interest Grouping ( EVI EEIG) aims to support and coordinate European vision research efforts. The company  Optobionics (Alan Chow, Vincent Chow) that developed the subretinal "Artificial Silicon Retina" or ASR went bankrupt in 2007 and its IP was acquired by Intelligent Medical Implants AG. However, the website was later restored with plans for a new company under the Optobionics name.

For retinal implants that make use of the optics of the eye, some people with retinal diseases may further benefit if they have their cataracts removed, receive a corneal transplant, have LASIK eye surgery applied, or have one of many other medical treatments or optical corrections (glasses, contact lenses) applied.

Electronic retinal implants may eventually be eclipsed by products of a more biochemical nature for which biocompatibility and high spatial resolution may be easier to achieve. In the not so distant future, treatment combinations with stem cell therapy, precursor cell therapy (transplanting developing retinal cells at later stages, closer to photoreceptor differentiation), gene therapy (optogenetics through gene-transfer, e.g., turning retinal nerve cells into photoreceptors by adding genes that support melanopsin or rhodopsin production through a virus carrying healthy gene copies or through some other carrier), retinal regeneration, eye surgery for immature retina transplants, and the use of growth factors and other pharmaceutical therapy might largely replace the use of electronic retinal implants, avoid any cell tissue overheating problems with electronic implants altogether and allow for restoring close to normal vision with retinal diseases like RP and AMD that only affect the rods and cones (light receptors) in the retina. In that sense, future electronic retinal implants may turn out to form a temporary technology in a rapidly evolving market, with limited commercial prospects if it becomes obsolete too soon, whereas The vOICe has the potential to find applications with a much broader range of causes of blindness, including many medically less tractable causes of blindness.

Gene therapy has been demonstrated for the first time in treating Steven Howarth, Vicki Duncan, Manuela Migliorati, twins Josalinda and Tommaso Ferraro and others for  Leber's congenital amaurosis or LCA, by "infecting" the eye with a virus carrying a healthy RPE65 gene copy. However, there is still uncertainty about the interpretation of these results, because it is conceivable that the significant eyesight improvements were not due to gene therapy, but rather to some side effects of performing surgery (which inevitably includes tissue responses to local injury). With the Optobionics implant too, improvements in vision were reported in areas around the chip implant that later on could not be explained from the designed implant properties, but had to be some sort of regenerative effect triggered by the insertion of a foreign object or by effects of surgery itself. With the above gene therapy trial, no "dummy" surgery was performed with everything the same except for leaving out the viral vector, and perhaps only the selective improvement of night vision can be held as an argument for the intended gene therapy effect. However, gene therapy remains a promising approach that may bring far higher resolution (visual acuity) than electronic retinal implants can bring in the foreseeable future.  RetroSense Therapeutics is one of the companies actively developing an optogenetic treatment for retinitis pigmentosa.

An ethical concern with early electronic retinal implants is that these may render recipients noneligible for later, more sophisticated remedies in case the early types would turn out to cause irreversible long-term damage to the retina.

More information about progress in retinal implant and cortical implant technology is presented every two years at  "The Eye and the Chip" World Conference on Artificial Vision in Detroit, USA, and also at the Annual Meeting of the Association for Research in Vision and Ophthalmology ( ARVO). For general medical progress, you can read recent publications about retinitis pigmentosa and macular degeneration.

You are invited to compare the actual level of artificial sight obtained via optoelectronic prosthetic retinas with that obtainable with The vOICe. Does an electronic retinal implant give you higher resolution than The vOICe? Does it give you a contiguous pixel map without gaps or stars-in-the-sky effect? Does it give you shading with more than two levels of brightness? What about reliability, biocompatibility and the percentage of defective pixels?

Further references:

N. R. B. Stiles, V. R. Patel and J. D. Weiland,  ``Multisensory perception in Argus II retinal prosthesis patients: Leveraging auditory-visual mappings to enhance prosthesis outcomes,'' Vision Research, Vol. 182, pp. 58-68, May 2021.

C. Erickson-Davis and H. Korzybska,  ``What do blind people 'see' with retinal prostheses? Observations and qualitative reports of epiretinal implant users,'' PLoS ONE, February 2021.

J. Kvansakul, L. Hamilton, L. N. Ayton, C. McCarthy and M. A. Petoe,  ``Sensory augmentation to aid training with retinal prostheses,'' Journal of Neural Engineering, June 2020.