Scientific publications based on use of The vOICe

The vOICe Home Page « The vOICe Home Page

Selected scientific publications based on use of or benchmarking of The vOICe sensory substitution technology for the blind, with a focus on neuroscience:

M. Kim, Y. Park, K. Moon and C. Y. Jeong, ``Impact of device and environment on visual-auditory sensory substitution: A comprehensive behavioral analysis using the vOICe algorithm,'' IEEE Access, June 2024. (Electronics Telecommunications Research Institute - ETRI, Korea) http://dx.doi.org/10.1109/ACCESS.2024.3419102
H.J. Kim, J. S. Kim, C. K. Chung, ``Visual mental imagery and neural dynamics of sensory substitution in the blindfolded subject,'' NeuroImage, 120621, May 2024. (Seoul National University, Seoul, Korea) https://doi.org/10.1016/j.neuroimage.2024.120621
J. Ruiz-Serra, J. White, S. Petrie, T. Kameneva and C. McCarthy, ``Learning scene representations for human-assistive displays using self-attention networks,'' ACM Trans. Multimedia Comput. Commun. Appl., March 2024. (Swinburne University of Technology, Hawthorn, Australia) https://doi.org/10.1145/3650111
L. Amaral, P. Thomas, Amir Amedi and Ella Striem-Amit, ``Longitudinal stability of individual brain plasticity patterns in blindness,'' bioRxiv 2023.11.01.565196, November 2023. (Georgetown University Medical Center, Washington, USA, and Reichman University, Herzliya, Israel) https://doi.org/10.1101/2023.11.01.565196
M. Kim, Y. Park, K. Moon and C. Y. Jeong, ``Deep learning-based optimization of visual-auditory sensory substitution,'' IEEE Access, February 2023. (Electronics Telecommunications Research Institute - ETRI, Korea) https://doi.org/10.1109/ACCESS.2023.3243641
A. S. Butorova, A. A. Naizagarinova, D. A. Tarasov and A. P. Sergeev, ``The vOICe visual-auditory sensory substitution technology in the depth perception task,'' 6th Scientific School Dynamics of Complex Networks and their Applications (DCNA), Kaliningrad, Russia, September 2022. (Ural Federal University, Ekaterinburg, Russia) https://doi.org/10.1109/DCNA56428.2022.9923163
G. Hamilton-Fletcher and K. C. Chan, "Auditory scene analysis principles improve image reconstruction abilities of novice vision-to-audio sensory substitution users," 2021 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5868-5871, November 2021. (New York University, New York, USA) https://doi.org/10.1109/EMBC46164.2021.9630296
S. Real and A. Araujo, ``VES: A mixed-reality development platform of navigation systems for blind and visually impaired ,'' Sensors, 21(18):6275, September 2021. (Polytechnic University of Madrid, Madrid, Spain) https://doi.org/10.3390/s21186275
https://www.mdpi.com/1424-8220/21/18/6275/htm
J. Pesnot Lerousseau, G. Arnold and M. Auvray, ``Training-induced plasticity enables visualizing sounds with a visual-to-auditory conversion device,'' Nature Scientific Reports, Vol. 11:14762, July 2021. (Aix-Marseille University, France, Caylar, France, and Sorbonne University, France) https://doi.org/10.1038/s41598-021-94133-4
J. Pesnot Lerousseau, G. Arnold and M. Auvray, ``Visualizing sounds: training-induced plasticity with a visual-to-auditory conversion device,'' bioRxiv 2021.01.14.426668, January 2021. (Aix-Marseille University, France, Caylar, France, and Sorbonne University, France) https://doi.org/10.1101/2021.01.14.426668
M. Kim, Y. Park, K. Moon and C. Y. Jeong, ``Analysis and validation of cross-modal generative adversarial network for sensory substitution,'' International Journal of Environmental Research and Public Health, 18(12), 6216, June 2021. (Electronics Telecommunications Research Institute - ETRI, Korea) https://doi.org/10.3390/ijerph18126216
J. Pesnot Lerousseau, G. Arnold and M. Auvray, ``Visualizing sounds: training-induced plasticity with a visual-to-auditory conversion device,'' bioRxiv 2021.01.14.426668, January 2021. (Aix-Marseille University, France, Caylar, France, and Sorbonne University, France) https://doi.org/10.1101/2021.01.14.426668
R. P. Kruger, F. de Wet, T. R. Niesler, ``Interactive image exploration for visually impaired readers using audio-augmented touch gestures,'' 24th International Conference Information Visualisation (IV2020), September 2020, (Stellenbosch University, South Africa, and CSIR, South Africa) https://doi.org/10.1109/IV51561.2020.00093
https://dsp.sun.ac.za/~trn/reports/kruger+dewet+niesler_iv2020.pdf
C. Jicol, T. Lloyd-Esenkaya, M. J. Proulx, S. Lange-Smith, M. Scheller, E. O'Neill and K. Petrini, ``Efficiency of sensory substitution devices alone and in combination with self-motion for spatial navigation in sighted and visually impaired,'' Frontiers in Psychology, Vol. 11:1443, July 2020. (University of Bath, UK, and Liverpool John Moores University, UK) https://doi.org/10.3389/fpsyg.2020.01443
J. Kvansakul, L. Hamilton, L. N. Ayton, C. McCarthy and M. A. Petoe, ``Sensory augmentation to aid training with retinal prostheses,'' Journal of Neural Engineering, June 2020. (University of Melbourne, Australia, Bionics Institute, Australia, Swinburne University of Technology, Australia) https://doi.org/10.1088/1741-2552/ab9e1d
M. J. Proulx, D. J. Brown, T. Esenkaya, J. B. Leveson, O. S. Todorov, S. Watson and A. A. de Sousa, ``Visual-to-auditory sensory substitution alters language asymmetry in both sighted novices and experienced visually impaired users,'' Applied Ergonomics, Volume 85, May 2020, 103072 and PsyArXiv, November 7 2019. (University of Bath, UK, and University of Queensland, Australia) https://doi.org/10.1016/j.apergo.2020.103072
https://doi.org/10.31234/osf.io/ys9xz
M. Uesaki, H. Ashida, A. Kitaoka and A. Pasqualotto, ``Cross-modal size-contrast illusion: Acoustic increases in intensity and bandwidth modulate haptic representation of object size,'' Nature Scientific Reports, Vol. 9:14440, October 2019. (Nanyang Technological University, Singapore, Kyoto University, Japan, and Ritsumeikan University, Osaka, Japan) https://doi.org/10.1038/s41598-019-50912-8
D. Hu, D. Wang, X. Li, F. Nie, Q. Wang, ``Listen to the Image,'' IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 7972-7981. (Northwestern Polytechnical University, China) http://openaccess.thecvf.com/content_CVPR_2019/papers/Hu_Listen_to_the_Image_CVPR_2019_paper.pdf
K. C. Chan, M. C. Murphy, J. W. Bang, J. Sims, J. Kashkoush and A. C. Nau, ``Functional MRI of sensory substitution in the blind,'' 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2018), Honolulu, USA. (New York University, USA, Mayo Clinic, USA, University of South Florida, USA and Korb and Associates, USA) https://doi.org/10.1109/EMBC.2018.8513622
https://ieeexplore.ieee.org/abstract/document/8513622
C. Graulty, O. Papaioannou, P. Bauer, M. Pitts and E. Canseco-Gonzalez, ``Hearing shapes: event-related potentials reveal the time course of auditory-visual sensory substitution,'' Journal of Cognitive Neuroscience, December 2017. (Reed College and University of California, USA) https://doi.org/10.1162/jocn_a_01210
https://sci-hub.se/10.1162/jocn_a_01210
PowerPoint at https://osf.io/7n34z/
S. Spagnol, S. Baldan and R. Unnthorsson, ``Auditory depth map representations with a sensory substitution scheme based on synthetic fluid sounds,'' 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP), Luton, UK, October 2017, pp. 1-6. (University of Iceland, Iceland and University of Venice, Italy) https://doi.org/10.1109/MMSP.2017.8122220
https://sci-hub.se/10.1109/MMSP.2017.8122220
K. C. Chan, M. C. Murphy, J. Kashkoush and A. C. Nau, ``Training-induced visual cortex activity correlates with duration of blindness and speed of performance during sensory substitution,'' Investigative Ophthalmology & Visual Science June 2017, Vol. 58 No. 8, 5627. ARVO Annual Meeting abstract, Baltimore, USA. (University of Pittsburgh, USA) http://iovs.arvojournals.org/article.aspx?articleid=2638265
N. Stiles, V. Chib and S. Shimojo, ``Auditory crossmodal plasticity can activate visual regions automatically and mildly deactivate natural vision,'' Meeting abstract presented at VSS 2016, Journal of Vision, September 2016, Vol. 16, 539. (California Institute of Technology, Pasadena, USA, and Johns Hopkins University, Baltimore, USA) https://doi.org/10.1167/16.12.539
https://jov.arvojournals.org/article.aspx?articleid=2550520
U. Hertz, D. Zoran, Y. Weiss and A. Amedi, ``fMRI dependent components analysis reveals dynamic relations between functional large scale cortical networks,'' bioRxiv Neuroscience, July 2016. (UCL Institute of Cognitive Neuroscience, London, UK, Google DeepMind, London, UK, Hebrew University of Jerusalem, Israel) https://dx.doi.org/10.1101/066282
https://www.seeingwithsound.com/extra/066282.full.pdf
D. J. Brown and M. J. Proulx, ``Audio-vision substitution for blind individuals: Addressing human information processing capacity limitations,'' IEEE Journal of Selected Topics in Signal Processing, Vol. 10, No. 5, pp. 924-931, August 2016. (University of Bath, UK) https://dx.doi.org/10.1109/JSTSP.2016.2543678
A. Pasqualotto and T. Esenkaya, ``Sensory substitution: the spatial updating of auditory scenes 'mimics' the spatial updating of visual scenes,'' Frontiers in Behavioral Neuroscience, Vol. 10:3389, 2016. (Sabanci University, Turkey, and University of Bath, UK) http://journal.frontiersin.org/article/10.3389/fnbeh.2016.00079/full
M. C. Murphy, A. C. Nau, C. Fisher, S.-G. Kim, J. S. Schuman and K. C. Chan, ``Top-down influence on the visual cortex of the blind during sensory substitution,'' NeuroImage, Vol. 125, pp. 932-940, January 2016. (University of Pittsburgh, USA, ...) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5536833/
https://dx.doi.org/10.1016/j.neuroimage.2015.11.021
A. C. Nau, M. C. Murphy and K. C. Chan, ``Use of sensory substitution devices as a model system for investigating cross-modal neuroplasticity in humans,'' Neural Regeneration Research, Vol. 10, No. 11, pp. 1717-1719, November 2015. (University of Pittsburgh, USA) https://dx.doi.org/10.4103/1673-5374.169612
N. Sigalov, S. Maidenbaum and A. Amedi, ``Reading in the dark: Neural correlates and cross-modal plasticity for learning to read entire words without visual experience,'' Neuropsychologia, Vol. 83, 2016. (Hebrew University of Jerusalem, Israel) https://dx.doi.org/10.1016/j.neuropsychologia.2015.11.009
https://docs.wixstatic.com/ugd/46b504_196ae3dbe7b54092a99f1da37e1278b6.pdf
M. J. Proulx, J. Gwinnutt, S. Dell’Erba, S. Levy-Tzedek, A. de Sousa, D. J. Brown, ``Other ways of seeing: From behavior to neural mechanisms in the online "visual" control of action with sensory substitution,'' Restorative Neurology and Neuroscience, Vol. 34, No. 1, pp. 29-44, November 2015. (University of Bath, UK, ...) https://dx.doi.org/10.3233/RNN-150541
N. Stiles and S. Shimojo, ``Auditory sensory substitution is intuitive and automatic with texture stimuli,'' Nature Scientific Reports, Vol. 5:15628, October 2015. (California Institute of Technology, Pasadena, USA) https://dx.doi.org/10.1038/srep15628
N. Stiles, ``Behavioral and fMRI measures of crossmodal plasticity induced by auditory sensory substitution,'' PhD thesis, California Institute of Technology, 2015. https://dx.doi.org/10.7907/Z9N29TZP
L. Reich and A. Amedi, ``'Visual' parsing can be taught quickly without visual experience during critical periods,'' Nature Scientific Reports, Vol. 5:15359, October 2015. (Hebrew University of Jerusalem, Israel) https://dx.doi.org/10.1038/srep15359
D. J. Brown, A. J. R. Simpson and M. J. Proulx, ``Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?'' Frontiers in Psychology, Vol. 6:1522, October 2015. (University of Bath, UK, ...) https://dx.doi.org/10.3389/fpsyg.2015.01522
D. J. Brown, ``Complexity, the auditory system, and perceptual learning in naïve users of a visual-to-auditory sensory substitution device,'' PhD thesis, Queen Mary University of London, 2015. https://qmro.qmul.ac.uk/jspui/handle/123456789/8985
https://qmro.qmul.ac.uk/jspui/bitstream/123456789/8985/1/Brown_David_PhD_020715.pdf
N. Stiles, Y. Zheng and S. Shimojo, ``Length and orientation constancy learning in 2-dimensions with auditory sensory substitution: The importance of self-initiated movement,'' Frontiers in Psychology, Vol. 6:842, June 2015. (California Institute of Technology, Pasadena, USA) https://dx.doi.org/10.3389/fpsyg.2015.00842
F. Bermejo, E. A. Di Paolo, M. X. Hüg and C. Arias, ``Sensorimotor strategies for recognizing geometrical shapes: a comparative study with different sensory substitution devices,'' Frontiers in Psychology, Vol. 6:679, June 2015. (National Technological University – Córdoba Regional Faculty, Argentina, ...) https://dx.doi.org/10.3389/fpsyg.2015.00679
C. Graulty, O. Papaioannou, P. Bauer, M. Stephens, J. Sheiman, M. Pitts and E. Canseco-Gonzalez, ``Hearing shapes: ERPs reveal changes in perceptual processing as a result of sensory substitution training,'' Cognitive Neuroscience Society (CNS) poster presentation, March 2015. (Reed College, USA) http://www.reed.edu/psychology/scalp/assets/conference%20files/Graulty_etal_CNS_poster_2015.pdf
D. J. Brown, A. J. R. Simpson and M. J. Proulx, ``Visual objects in the auditory system in sensory substitution: How much information do we need?,'' Multisensory Research Vol. 27, 2014, pp. 337-357. (Queen Mary University of London, UK, ...) https://dx.doi.org/10.1163/22134808-00002462
C. Graulty, O. Papaioannou, P. Bauer, M. Pitts and E. Canseco-Gonzalez, ``Electrophysiological dynamics of auditory-visual sensory substitution,'' Vision Sciences Society (VSS) poster presentation, May 2014. (Reed College, USA) http://www.reed.edu/psychology/scalp/assets/conference%20files/Graulty_VSS_Poster_2014.pdf
E. Striem-Amit and A. Amedi, ``Visual Cortex extrastriate body-selective area activation in congenitally blind people "seeing" by using sounds,'' Current Biology, Vol. 24, No. 6, pp. 687-692, March 2014. (Hebrew University of Jerusalem, Israel) https://dx.doi.org/10.1016/j.cub.2014.02.010
http://www.sciencedirect.com/science/article/pii/S0960982214001481
U. Hertz and A. Amedi, ``Flexibility and stability in sensory processing revealed using visual-to-auditory sensory substitution,'' Cerebral Cortex, Vol. 25, No. 8, pp. 2049-2064, 2014. (Hebrew University of Jerusalem, Israel) https://dx.doi.org/10.1093/cercor/bhu010
http://cercor.oxfordjournals.org/content/25/8/2049.full
N. R. B. Stiles and S. Shimojo, ``Sensory substitution: A new perceptual experience,'' Oxford Handbook of Perceptual Organization, Oxford University Press, 2014. (California Institute of Technology, Pasadena, USA) https://dx.doi.org/10.1093/oxfordhb/9780199686858.013.050
http://www.gestaltrevision.be/pdfs/oxford/Stiles&Shimojo-Sensory_substitution_A_new_perceptual_experience.pdf
D. J. Brown and M. J. Proulx, ``Increased signal complexity improves the breadth of generalization in auditory perceptual learning,'' Neural Plasticity, 879047, November 2013. (University of Bath, UK) https://doi.org/10.1155/2013/879047
E. Striem-Amit, ``Neuroplasticity in the blind and sensory substitution for vision,'' PhD thesis, Hebrew University of Jerusalem, November 2013. https://www.seeingwithsound.com/extra/Ella%20Striem-Amit%20-%20PhD%20thesis.pdf
A. Haigh, D. J. Brown, P. Meijer and M. J. Proulx, ``How well do you see what you hear? The acuity of visual-to-auditory sensory substitution,'' Frontiers in Cognitive Science, Vol. 4:330, June 2013. (Queen Mary University of London, UK, ...) https://dx.doi.org/10.3389/fpsyg.2013.00330
M. J. Proulx, D. J. Brown, A. Pasqualotto and P. Meijer, ``Multisensory perceptual learning and sensory substitution,'' Neuroscience and Biobehavioral Reviews, Vol. 41, April 2014, pp. 16-25. (University of Bath, UK, ...) https://dx.doi.org/10.1016/j.neubiorev.2012.11.017
https://sci-hub.se/10.1016/j.neubiorev.2012.11.017
E. Striem-Amit, L. Cohen, S. Dehaene and A. Amedi, ``Reading with sounds: sensory substitution selectively activates the visual word form area in the blind,'' Neuron, Vol. 76, No. 3, pp. 640-652. (Hebrew University of Jerusalem, Israel, ...) https://dx.doi.org/10.1016/j.neuron.2012.08.026
http://www.cell.com/neuron/pdf/S0896-6273(12)00763-5.pdf
http://www.sciencedirect.com/science/article/pii/S0896627312007635
N. Stiles, V. Chib and S. Shimojo, ``Behavioral and fMRI measures of "visual" processing with a sensory substitution device,'' Meeting abstract presented at VSS 2012, Journal of Vision, August 2012, Vol. 12, 703. (California Institute of Technology, Pasadena, USA) https://doi.org/10.1167/12.9.703
https://jov.arvojournals.org/article.aspx?articleid=2141350
E. Striem-Amit, O. Dakwar, L. Reich and A. Amedi, ``The large-Scale organization of "visual" streams emerges without visual experience,'' Cerebral Cortex, Vol. 22, No. 7, pp. 1698–1709, July 2012. (Hebrew University of Jerusalem, Israel) https://doi.org/10.1093/cercor/bhr253
https://academic.oup.com/cercor/article/22/7/1698/295478
E. Striem-Amit, M. Guendelman and A. Amedi, ``'Visual' Acuity of the Congenitally Blind Using Visual-to-Auditory Sensory Substitution,'' PLoS ONE, Vol. 7, No. 3, March 2012, e33136. (Hebrew University of Jerusalem, Israel) https://dx.doi.org/10.1371/journal.pone.0033136
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0033136
J.-K. Kim and R. J. Zatorre, ``Tactile-auditory shape learning engages the Lateral Occipital Complex,'' Journal of Neuroscience, Vol. 31, No. 21, pp. 7848-7856, May 2011. (Montreal Neurological Institute, McGill University, Canada) https://dx.doi.org/10.1523/JNEUROSCI.3399-10.2011
http://www.jneurosci.org/content/31/21/7848
E. Striem-Amit, O. Dakwar, U. Hertz, P. Meijer, W. Stern, A. Pascual-Leone and A. Amedi, ``The Neural Network of Sensory-Substitution Object Shape Recognition,'' Functional Neurology, Rehabilitation, and Ergonomics, Vol. 1, No. 2, pp. 271-278. Nova Science Publishers, 2011. (Hebrew University of Jerusalem, Israel, ...) http://brain.huji.ac.il/publications/Ella_ergo2011.pdf
D. Brown, T. Macpherson and J. Ward, ``Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device,'' Perception, Vol. 40, No. 9, pp. 1120-1135, 2011. (Queen Mary University of London, UK, ...) https://dx.doi.org/10.1068/p6952
https://sci-hub.se/10.1068/p6952
J.-K. Kim, ``Sensory substitution learning using auditory input: Behavioral and neural correlates'', PhD thesis, Montreal Neurological Institute, McGill University, 2010. http://digitool.library.mcgill.ca/webclient/DeliveryManager?pid=96695
J. Ward and P. Meijer, ``Visual Experiences in the Blind induced by an Auditory Sensory Substitution Device,'' Consciousness and Cognition, March 2010. (University of Sussex, Brighton, UK, ...) https://dx.doi.org/10.1016/j.concog.2009.10.006
https://sci-hub.se/10.1016/j.concog.2009.10.006
https://www.seeingwithsound.com/extra/cc2009_preprint.pdf
J.-K. Kim and R. J. Zatorre, ``Can you hear shapes you touch?'', Experimental Brain Research, Vol. 202, No. 4, pp. 747-754, February 2010. (Montreal Neurological Institute, McGill University, Canada) https://dx.doi.org/10.1007/s00221-010-2178-6
https://sci-hub.se/10.1007/s00221-010-2178-6
L. B. Merabet, L. Battelli, S. Obretenova, S. Maguire, P. Meijer and A. Pascual-Leone, ``Functional recruitment of visual cortex for sound encoded object identification in the blind,'' Neuroreport, Vol. 20, No. 2, pp. 132-138, January 2009. (Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA, ...) https://dx.doi.org/10.1097/WNR.0b013e32832104dc
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3951767/
A. Amedi, W. Stern, E. Striem, U. Hertz, P. Meijer and A. Pascual-Leone, ``A what/where visual-to-auditory sensory substitution fMRI study: can blind and sighted hear shapes and locations in the visual cortex?,'' 31st European Conference on Visual Perception (ECVP 2008), August 2008. (Hebrew University of Jerusalem, Israel, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA, ...) http://www.perceptionweb.com/abstract.cgi?id=v080153
M. J. Proulx, P. Stoerig, E. Ludowig and I. Knoll, ``Seeing 'Where' through the Ears: Effects of Learning-by-Doing and Long-Term Sensory Deprivation on Localization Based on Image-to-Sound Substitution,'' PLoS ONE, Vol. 3, No. 3, March 2008, e1840. (University Düsseldorf, Germany) https://dx.doi.org/10.1371/journal.pone.0001840
J.-K. Kim and R. J. Zatorre, ``Generalized learning of visual-to-auditory substitution in sighted individuals,'' Brain Research, Vol. 242, pp. 263-275, 2008. (Montreal Neurological Institute, McGill University, Canada) https://dx.doi.org/10.1016/j.brainres.2008.06.038
https://www.zlab.mcgill.ca/publications/docs/Kim_and_Zatorre_2008.pdf
A. Amedi, W. Stern, J. A. Camprodon, F. Bermpohl, L. Merabet, S. Rotman, C. Hemond, P. Meijer and A. Pascual-Leone, ``Shape conveyed by visual- to-auditory sensory substitution activates the lateral occipital complex,'' Nature Neuroscience, Vol. 10, No. 6, pp. 687 - 689, June 2007. (Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA, ...) https://dx.doi.org/10.1038/nn1912
http://brain.huji.ac.il/publications/Amedi_at_al_vOICe_Nature_Neuroscience07.pdf
M. Auvray, S. Hanneton and J. K. O'Regan, ``Learning to perceive with a visuo-auditory substitution system: Localisation and object recognition with 'The vOICe','' Perception, Vol. 36, No. 3, pp. 416-430, 2007. (University of Oxford, UK, ...) https://dx.doi.org/10.1068/p5631
https://sci-hub.se/10.1068/p5631
P. B. L. Meijer, ``An Experimental System for Auditory Image Representations,'' IEEE Trans. Biomedical Engineering Vol. 39, No. 2, pp. 112-121, Feb 1992. (Philips Research, Eindhoven, The Netherlands) https://dx.doi.org/10.1109/10.121642
https://www.seeingwithsound.com/voicebme.html

Copyright © 1996 - 2024 Peter B.L. Meijer