The vOICe and 3D Games

This web page is no longer actively maintained and may therefore be partially obsolete.

3D Games for the Blind Technical feasibility of accessing 3D games

Click to hear The vOICe Half-Life MP3 soundscape (18K)

Mainstream 3D games are often considered completely inaccessible to totally blind people, but The vOICe technology now begins to change that. The vOICe can sound the visual content presented on the computer screen by 3D games. In a way, you become your own sighted avatar.

This is really stretching the limits, because games often take full control of the computer, "down to the metal", and may or may not tolerate that The vOICe keeps running and functioning in the background. Still, you can now view a number of "inaccessible" 3D games in sound via The vOICe! The recipe for this will be outlined below. No camera or other video capture device is needed here, as everything is handled in software. The vOICe captures the screen content, and will even automatically adapt to any changes in screen resolution as often applied by 3D games when they start up. In principle, you could even read graphical text in the scene through the optional OCR add-on, but results here depend strongly on advances in the quality of third-party OCR engines for use with The vOICe. Therefore, depending on the game, you may still need some sighted assistance to get started, while learning the main controls and the basics of visual interpretation of the game scenery.

Yet The vOICe approach itself is completely general and in principle applies to any unmodified 3D game as developed for the sighted.

Just give it a try and find out if it works with your 3D game. No guarantees here, but a number of 3D games for the sighted were indeed found to work with the screen sonification feature of The vOICe, including for instance Microsoft Train Simulator. In another example, one early blind user reportedly took off, flew and landed a plane with Microsoft Flight Simulator in combination with The vOICe. (Never try this type of synthetic vision for "flying blind" in the real world!) Yet another blind user found that in the accessible game Terraformers he was now able to distinguish where walls, doors, and the other objects were without using the built-in functions of the game. Still another blind user reported that in the accessible game AudioQuake, The vOICe made him aware of facing a wall before the built-in echolocation informed him of this.

The basic recipe is simply as follows:

1. Start The vOICe for Windows (Control Alt v), and press the function key combination Shift F9 to enter full screen sonification.

   You should now be hearing whatever is displayed on your screen. A small area of your screen will still be covered by The vOICe window.

2. Minimize (Alt space n) The vOICe for Windows window.

   This is needed to prevent that The vOICe window occludes part of your game view, because normally The vOICe window stays in front of all windows. Later on, Control Alt v may be used to restore the normal window state for The vOICe to access its various controls.

3. Start your 3D game, hoping for the best...

You can experiment with the above procedure in combination with any screen-based computer game, such as Quake, Second Life and many others. Much the same technology can also be applied to access purely visual VRML/X3D scenes and virtual worlds, as is explained on the VRML/X3D for the Blind page, and it may also be used to enjoy the graphics of already accessible games such as  Terraformers of Thomas Westin and others, and  AudioQuake (AGRIP Project) of Matthew Tylee Atkinson and Sabahattin Gucukoglu. It may also be combined with the work of  Eelke Folmer and  Gareth White (blindsecondlife) on Second Life, to have a training environment for The vOICe where virtual objects and obstacles are effectively labelled with their spoken names, and with distances indicated through a virtual sonar. These combined technologies can thus play a role in various forms of virtual wayfinding training.

On June 27, 2009, The vOICe was demonstrated by Tali Walmer and others at the  Helen Keller Day in Second Life ( SLURL), using the 3D virtual environment sonification features of The vOICe, as also explored by Louise Later. Both Tali Walmer and Louise Later (Louise Nicholson) have low vision in real life. Louise Later: ``I noticed that Tali Walmer, who also can't see well, was turning beautifully at the edge and even sitting in chairs. I asked her how she was doing this miracle in SL. She said that she was using 'The vOICe' application.'' Because of the white background inside the room, The vOICe is here best used with inverse video (function key F5). Note: when last checked all props had been removed from the training room, such that it may now show as only a structureless white space.

The primary focus of The vOICe technology remains on rendering live camera views into sound, using a wearable setup with a head-mounted camera, thus making for what can also be considered a challenging and fully immersive reality game, the "game of sight", via sound. For more information about accessibility and (alternate, reality) games, you can visit the websites of the  Alternate Reality Games Special Interest Group (now archived) of the Independent Game Developers Association (ARG SIG IGDA), the  Alternate Reality Gaming Network (ARGN) and associated  Unfiction/Unforums, and the  Game Accessibility Special Interest Group (Game Accessibility SIG) of the International Game Developers Association (IGDA).