Sonar extension for The vOICe

This web page is obsolete and will no longer be maintained.

The vOICe seeing-with-sound technology provides blind users with camera-based visual information through sound. Especially while still learning to "see" with sound, it can be difficult to reliably tell foreground from background, and a sonar extension may help detect nearby objects and obstacles for increased safety and comfort. A prototype sonar extension for use in combination with The vOICe is currently under development in a cooperation with Dr. Sergey Ershov (Ершов, Сергей Олегович) of the  Baltic State Technical University (BSTU "VOENMEKH") and  OOO Scientific-Manufacturing Organization Sonar (ООО «НПО “Сонар”») in St. Petersburg, Russia.

Sonar transducers, mounted on video sunglasses with a hidden camera, track the viewing direction of the blind user, and the ultrasonic feedback signal is used to map the distance to the nearest object(s). The distance mapping is further processed by The vOICe for Windows software for proper auditory display in combination with the camera-based soundscapes, avoiding ambiguities in the semantics of the resulting sounds, and bepping with a frequency representing the distance. The vOICe for Windows runs on a notebook PC and communicates with the sonar device via a regular parallel port connection, with an additional USB connection only for drawing power, if needed. The photograph below shows the prototype sonar device.

Prototype sonar extension for The vOICe

The sonar extension would normally be combined with the same kinds of video sunglasses that are already in use with The vOICe for Windows. These video sunglasses contain a tiny hidden camera that feeds video to The vOICe, typically via a USB video capture device. The two small sonar transducers can be attached to the legs of the video sunglasses, as indicated in the photograph. One transducer transmits the ultrasound signal, while the other transducer receives the ultrasound return signal as bounced off and reflected by any nearby objects within the relatively wide sonar beam. The time delay between transmitted and received signal is proportional to the distance, and the digitized distance codes are used by The vOICe to give user feedback on nearby objects. In combination with the live camera view this further supports perception of 3D location information. The following photograph gives a close-up frontal view of the prototype sonar setup.

The sonar extension would normally be combined with video sunglasses as used with The vOICe

The prototype sonar extension will be used to guide further technical development as well as be used in initial evaluation studies with blind users for further refinements based on their feedback. Interested distributors and other potential partners may contact the developers via e-mail at the account for further information. E-mail sent to this account is automatically forwarded to the developer of the sonar extension for The vOICe in St. Petersburg, Russia, as well as to the developer of The vOICe. Other sonar ETA (Electronic Travel Aid) product information from OOO Scientific-Manufacturing Organization Sonar is available in  Sonar_ETA.doc (Microsoft Word format).

Another combination of a sonar extension with video sunglasses and earpieces as used with The vOICe

Further information translated into Russian is available for a BBC News article about The vOICe titled  "Слепых учат видеть ушами" ( "Blind 'see with sound'") and for a number of user accounts,  "Что слепые пользователи говорят о vOICe" ("What blind users say about The vOICe").

Update and legal warning

Since January 1, 2010, production, sale or purchase of camera glasses for the blind may have become illegal in Russia as a result of a  revised criminal law reading "Уголовный кодекс Российской Федерации… 3. Незаконные производство, сбыт или приобретение специальных технических средств, предназначенных для негласного получения информации". The prototype devices shown on this page were created before 2005. We can only hope that suitable exceptions will be defined for the perfectly legitimate uses of The vOICe by blind people in Russia. See also the online discussion  "3 года тюрьмы за дополненную реальность" (3 years in prison for augmented reality). Note that The vOICe in normal use does not store or broadcast any video, but merely acts as the blind user's "eye", while the camera should be concealed for a normal non-stigmatizing appearance.

Literature

S. O. Ershov and P. Meijer (С. О. Ершов и П. Майер),  ``Системы технического зрения с невизуальным представлением пространственной информации,'' Известия вузов. Приборостроение, Т. 51, № 1. С. 42—47, January 2008 (in Russian),

Abstract - Рассмотрены системы технического зрения, позволяющие человеку воспринимать пространственную информацию посредством слуха или осязания и применяемые в технических средствах ориентирования для слепых и работников аварийно-спасательных служб. Приведены краткие сведения по истории и текущему состоянию вопроса. Предложен вариант классификации рассматриваемых систем. Более подробно рассмотрен принцип действия наиболее информативных средств ориентирования, их перспективы и пути усовершенствования. Обсуждены проблемы, связанные с внедрением приборов данного класса в серийное производство. Ил. 3, библиогр. 3.

S. O. Ershov, P. Meijer and E .L. Baki-Borodov (С.О. Ершов, П. Мейер и Е. Л. Баки-Бородов),  ``Локальные и глобальные программно-аппаратные средства позиционирования и навигации для слепых’’ (Local and global software means of positioning and navigation for the blind), Информатизация и связь, № 1, 2011 (in Russian).

Abstract - статье проводится сравнительный анализ возможностей использования различных диапазонов волнового спектра (ультразвукового, инфракрасного, радиодиапазона /включая средства космической связи/), для обеспечения позиционирования и навигации слабовидящих и слепых. Приводятся сведения о конкретных отечественных и зарубежных разработках индивидуальных средств технического зрения и преобразования результатов локальной ультразвуковой и оптической локации объектов окружающей среды (на открытых городских пространствах и в помещении) в полифонический звуковой сигнал или речевую форму для обеспечения пространственной ориентации пользователей с существенными анатомо-физиологическими ограничениями по зрению.

Keywords: позиционирование, навигация, ГЛОНАСС, GPS, ультразвук, оптическое излучение, техническое зрение, индивидуальные технические средства дальнометрия, тактильное средство, ультразвуковое средство реабилитации, ультразвуковые стереофонические очки, видеокамера, речевой доступ, слепые, слабовидящие.

Note: The vOICe technology is being explored and developed under the Open Innovation paradigm together with R&D partners around the world.