« The vOICe Home PageOther projects based on or related to The vOICe approach:
BrushTone, an accessible Paint Tool for the visually impaired that allows users to view, modify, and create images purely using sound. It includes a window scan function based on The vOICe.
After-Sight-Model-1 GitHub repository, which includes raspivoice, a version of The vOICe for the Raspberry Pi, and teradeep, a deep learning neural network for visual object recognition.
Python (I2S*.zip), further using Pyglet, PyAL, OpenAL and Numpy.
raspivoice, a version of The vOICe for the Raspberry Pi. More about this in the Raspberry Pi forum thread
Sight for the Blind for <100$. Mike Mcwilliams developed a Raspberry Pi device for The vOICe.
"EyeMusic: Hearing colored shapes"
for iPhone, iPod Touch and iPad.
OpenCV
and
RtAudio.
"Voice vision"
for iPhone, iPod Touch and iPad.
SoundsOfImages,
described as "Can you hear the Image? Tool that allows you to transform image into audible sounds".
Wavy,
a flexible visual-to-auditory sensory substitution system written in Python.
audioScreen, an experimental program
for blind users of Windows 7 touch screens, and based on The vOICe mapping.
Pure Data
(Pd patch) implementation of image to sound conversion,
voice.pd.
Also in 2010, he created a "Customisable image sonifier" written in C#,
named Polyglot Framework for Sensory Substitution Devices.
Katarzyna Zarnowiec,
student Automation and Robotics at the AGH University of Science and Technology in Krakow, Poland, and Frederico Contente,
student at Tampere University of Technology, Finland, implemented a Matlab program
"Sound Steganography",
using a mapping related to that used by The vOICe.
Encoding an image to sound.
OpenSonify
for combining webcam, PC and headphones.
Hans Petter Selasky
developed an image-to-sound mapping similar to The vOICe but based on noise filtering
(subtractive synthesis instead of additive synthesis).
"Technology to help blind people: Image restitution by sonorous signals" (PDF file)
is provided here with his permission.
Malika Auvray
of COSTECH at the University of Technology of Compiègne and
Kevin O'Regan
of the Laboratory for Experimental Psychology of the University of Paris 5 disclosed some of their work on the "Vibe"
as a system for instantaneous visual feedback, developed by Sylvain Hanneton of the University of Paris 5.
It maps brightness to loudness, height to pitch, and lateral position to stereo sound, somewhat like The vOICe
running in its optional "All at once" mode (menu Options | Video Rate). More information is available on
the SourceForge
thevibe
project page, and the web pages of
"Visuo-auditory sensory substitution for mobility assistance: testing TheVIBE".
Primatop
(Fabrice Pajak) started marketing their MIDI-based VisioPlayer System for audio-visual sensory substitution for the blind.
Some of the hardware options for camera input and visual display seemed based on an Age Tech RF-ZLCD30E wireless camera with
LCD receiver ("VisioMonitor" option) and a ZTV 830G Mini Wireless PC Camera Kit, combined with Philips SBC-HC8441 FM wireless
stereo headphones.
"DORA:
Digital Object Recognition Audio-Assistant For The Visually Impaired").
"Tyflos".
The focus of the Tyflos/iCare project seems to be more on attempts to recognize and give verbal descriptions
of things, similar to what can be done via The vOICe's Mobile OCR for the Blind interface,
while the main focus of The vOICe is on providing the "raw" visual information via soundscapes.
A. R. O'Hea, ``Optophone Design: Optical-to-Auditory Vision Substitution for the Blind,'' Ph.D. thesis, The Open University, UK, 1994.Adrian O'Hea appears to have independently discovered auditory mappings that are similar to The vOICe mapping (which he calls the Cartesian Piano Transform). He proposed a number of variations, like the use of a polar coordinate system to obtain a kind of artificial fovea. The focus of Michael Capp's Ph.D. work later shifted towards the independent development of stereo vision options for depth mapping, in combination with "cartoon mapping" for the preservation of visual texture. In 2000, Michael Capp received his Ph.D. on ``Alternative approaches to optophonic mappings'' from Leicester University, UK, with Peter Meijer acting as invited External Examiner.
Virtual Acoustic Space
(Espacio Acústico Virtual, or EAV) project
of the Institute of Astrophysics of the Canary Islands (IAC) and the
University of La Laguna, Tenerife, Spain.
"nanovoice".
Very interesting and original work!
John Cronly-Dillon,
a neurobiologist at the Department of Optometry and Vision Sciences at the University of
Manchester (UMIST), UK. The broadcast showed examples in which the basic characteristics of
transforming shapes into music for hearing images appeared identical to those employed by The vOICe
(and even to those employed by the pianola and optophone): left-to-right scanning, with pitch depending
on elevation, and with a vertical line segment resulting in a chord with all tones sounding
simultaneously, as with the four vertical edges in the MIDI sample
for the box shown on the left. Cronly-Dillon's implementation was a computer program without
live visual input, but he mentioned plans for a future portable system with a camera. According
to an article in The Guardian UK of September 22, 2000, it will be called "SmartSight" and have
the looks of a Star Trek visor. ``Backing up claims that this is an innovation from the realms
of science fiction, the new gadget is almost identical to that worn by blind character Geordi
la Forge on Star Trek's "The Next Generation".'' [...] ``The scientists say an early prototype
will be ready for testing before the end of this year. A battery pack would be worn at the waist,
while the "eye" of the device would either he a hand-held video camera, or two cameras fitted
into a visor worn on the head.''
See also the archived website of the associated
SmartSight Limited startup venture with
John Cronly-Dillon, Krishna Persaud and David Stead. Smartsight Limited was registered at
Companies House on July 15, 1999, under company number 03810177.
"Music" score with MIDI sample for circle, triangle and square
A demonstration was given at the
RNIB Techshare 2001
Conference in Birmingham, UK.
The Sonic Eye for walking and reading
``A voice print represents sounds as images. High pitch sounds appear as marks near the top of the image; low pitch sounds appear as marks near the bottom of the image. Time is a vertical line moving at a constant speed from left to right. Thus a voice print can convert any sound into an image. The sonic eye does the reverse to convert images into sound. The sonic eye "sees" a thin vertical window wherever it is pointed and makes a sound combining high pitch for objects near the top of the window and low pitch for objects near the bottom as the thin window is swept across objects of interest. The user listens to these sounds and learns about the visual environment.'' [Harry Reid, 1998]
ThalesScope,
which is supposed to convert sight into sound. Other than this, little factual
data is available about it at the time of this writing, and it is also unclear
where it would differ from existing The vOICe technology.
ThalesScope Limited is headed by Isaiah W. Cox, and owned in part by
Thales Resources, and in part by Tom Karnes. Borealis Technical Limited
does the build, development, and test work.The latest information now suggests that this project might have been discontinued or at least is suffering from serious delays.
Note: The vOICe approach was originally published in the IEEE Transactions on Biomedical Engineering, Vol. 39, No. 2, pp. 112-121, Feb 1992: P.B.L. Meijer, ``An Experimental System for Auditory Image Representations.'' This paper was next selected for reprint in the 1993 IMIA Yearbook of Medical Informatics, pp. 291-300. Awarded U.S. Patent 5097326, on an "image-audio transformation system", filed July 27, 1990: ``In a device for converting visual images into representative sound information especially for visually handicapped persons an image processing unit is provided with a pipelined architecture with a high level of parallelisum. An image is scanned in sequential vertical scanlines and the acoustical representatives of the scanlines are produced in real time. Each scanline acoustical representation is formed by sinusoidal contributions from each pixel in the scanline, the frequency of the contribution being determined by the position of the pixel in the scanline and the amplitude of the contribution being determined by the brightness of the pixel.''