The vOICe for Windows

« The vOICe Home Page
« The vOICe for Windows (introduction and download)

Advanced options: Once you have grown accustomed to the basic operation of The vOICe for Windows, you may wish to make use of some of the ``advanced options'' as referred to on the main The vOICe for Windows page, including

• A noise suppression option toggled by pressing key 'N': it aims to remove irrelevant fine granularity without removing other detail or strong textures. It may thereby help to mentally absorb the view as a whole as well as help distinguish its constituent parts.

• A low light boost option toggled by pressing key 'L': it can help to perceive items in dimly lit indoor situations, but a downside is that you "see" details that may be irrelevant depending on the task at hand. E.g. dust can be meaningful when you are cleaning, but distracting when you want to locate and pick up an object on a dark but dusty background.

• Scan direction can be toggled between left-to-right (default) and right-to-left by pressing the key combination Control Alt L. Right-to-left scanning can be useful when visually reading Hebrew and Arab print.

• Experimental 3D motion analysis buttons (F1 and Alt F1): these let you audibly highlight moving or approaching objects and shapes, corresponding to real-time scene changes or imminent collisions (hazard detection and collision warning to serve collision avoidance). The monocular collision threat analysis hashes those parts of the camera view where an imminent collision is suspected. Apart from this monocular visual threat warning support, software support for binocular vision through a stereo camera is available for depth mapping segmentation, auditory disparity synthesis and other uses.

• Two slow-motion buttons (F3 and Control Alt F3): these let you hear more detail by using a doubled or quadrupled scan-time. This is particularly useful for analyzing static images or scenes, in which case you can afford to sacrifice the tracking of movements.

• Three fast-motion buttons (Shift F3, Alt F3 and Control F3): these let you hear movements more accurately at the expense of detail, by using a reduced scan-time. This is particularly useful for highly dynamic scenes in which movement tracking is considered more important than resolution. Shift F3 gives twice the normal speed, Alt F3 gives four times the normal speed, and Control F3 even gives eight times the normal speed. With a normal speed of one video frame per second this then gives two, four, or eight frames per second, respectively.

• A digital zoom button (F4): lets you hear more detail, by zooming into a scene without changing the scan-time.

  

  This option enlarges the middle part of the camera image just like a zoom lens would, but without requiring such a lens! F4 magnifies two times, Shift F4 four times, Control F4 eight times and Shift Control F4 sixteen times. (Be careful not to hit Alt F4 accidentally since this is the usual key combination to quit the program.)

• An negative video button (F5): uses negative images, thus giving better sound contrast in very bright visual environments, e.g., with lots of snow or sand.

  

  Normally, small dark objects on a bright background can be difficult to hear, but this option highlights objects in exactly this situation, by turning them into bright objects on a dark background.

• Two contrast enhancement buttons (F6 and Control F6): these increase the auditory contrast to highlight image items that would otherwise be much harder to hear. Currently, the default state at start-up is to have contrast enhancement turned on.

  Technical notes: in addition to this contrast enhancement feature, it can be useful to set a high video contrast via the menu Edit | Audio/Video Preferences | Video Source... | Contrast. The best choice may vary with brand and type of camera, but generally a high contrast makes it easier to discern the main items in a sonified image. With the Creative Labs Video Blaster WebCam I the maximum contrast value of 100 appeared to be a good choice. Also, to reduce video noise ("snow"), and thereby improve sound quality, one can either disable video compression altogether or choose a high compression quality (close to 100) from the Edit | Audio/Video Preferences | Video Source... | Capture Compression menu. There is a trade-off with CPU load, though, so completely turning the capture compression off may only work well on very fast PC's (200 MHz Pentium or better). A compression quality of about 70 may be a good compromise.

• An edge enhancement button (F7): highlights edges that might otherwise be hardly noticeable, while reducing the loudness of uniformly shaded areas. Edge detection can help to identify and interpret shapes and visual perspective.

  

• A math function plot button (F8): opens a dialog that allows you to make auditory function plots in combination with a screen reader. All the usual mathematical functions such as sin(x), sqrt(x) and exp(x) are supported using a common C-like syntax.

  This function allows The vOICe to be used as an auditory graphing calculator in combination with a suitable screen reader. Of course you can also use the camera to hear printed graphs, for instance from a book.

• A GUI sonification button (F9): turns the area around your mouse pointer on your computer screen into sound. This then overrules any direct video input from your camera. This option can be combined with the zoom and negative video options for best results, and provides access to any graphical item of the graphical user interface (GUI) as displayed on your screen, including window borders, graphs, image maps and graphical buttons. Shift F9 gives an overview of the entire screen content. Alt F9 sonifies the active window. Control F9 provides a special sonification mode for the single pixel under the mouse pointer, using two tones to sound horizontal position and vertical position, but otherwise mapping brightness to loudness as usual.

  

  Full screen GUI sonification may also be used for access to VRML/X3D virtual worlds or to hear the scenery in modern 3D graphics games.

  Color Sensing and Identification
  The quality of your color camera is decisive for good and accurate color sensing and identification, because camera color may deviate from the color perception of the sighted. Also make sure that you are using a 65,000 color (16-bit) or higher display mode, or else you will only get "grey" detection!	Any little green men?
  Shift+Alt+g for green	Shift+Alt+r for red

• A color identification button (F10): the talking color probe picks and speaks the color detected in the center of the camera view. Now you know whether the apple you are about to eat is yellow, green or red, and you can check the dominant colors of your clothing. Recognized colors include (dark, normal, and light) red, green, blue, cyan, yellow, orange and magenta, as well as combination colors. Black, grey and white are also identified.

  Furthermore, you can make use of color filtering keys. Are you perhaps searching for little green men? You can detect and pick out items of a specified color by pressing the Shift-key and Alt-key in combination with the first letter of the desired color. For instance, Shift+Alt+g selects and sounds only green items, and Shift+Alt+r selects any red items. Other selectable colors are blue, cyan, yellow, orange and magenta. Pressing a color filter key combination for a second time toggles the color filtering off again ("white" filter). Moreover, pressing Shift+Alt+s toggles a built-in skin color detection mode that helps you locate faces and hands. Note though that other materials with skin-like colors, such as wood, may easily be mistaken for human skin.

  Technical notes: the color sensor only supports "grey" detection in a 256 color (8-bit) screen display mode, so switch to a 65,000 color (16-bit) or higher display mode to get the full functionality: you may change these settings via Control Panel | Display | Settings. If this tab appears disabled, there probably was another tab installed for your particular graphics adapter. For instance, for a Matrox Millennium graphics card one should go to Control Panel | Display | MGA Settings to change the display mode.

  
  Shift+Alt+s: Filter skin color to detect humans on bright and cluttered visual backgrounds

• An exercise button (F11): an automatic pattern generator is available for training purposes. It allows you to hear the soundscapes of randomly placed rectangles, circles, triangles and line segments.
  

  The generated patterns can be used for exercising your conscious mental analysis: what shapes (objects) do you hear at what positions? For simple patterns like the one shown on the left this will even allow for training through self-study without tactile or sighted reference feedback on correctness: your brain will build a self-consistent map. Beware though that normal human beings, blind or sighted, have a very poor memory for meaningless random patterns, so you should not expect to be able to mentally grasp the positions of more than a few shapes or objects at a time from these soundscapes! In the exercise mode, you can activate manual pattern updating by pressing the space bar. The arrow keys will then move a shape, and the plus and minus keys on the numeric keypad will select a shape. If desired, speech can be turned off via Alt F10.

  By default, two filled rectangles of varying brightness are randomly positioned, with the pattern changing either automatically every three soundscape scans, or manually after the space bar has been pressed. Other settings can be configured via the Edit | Exercise Preferences | Randomly Placed Shapes menu.

  Further usage notes: with the default two bright rectangles of the exercise mode, you get two sound bursts, with duration depending on their width, pitch depending on their height and elevation, and loudness depending on brightness. Everytime you press the space bar, you get a new random size and position for two filled rectangles. Subsequently, you can move the currently selected shape around by using your arrow keys, and you can use the plus and minus keys on your numeric keyboard to select another shape to move around. The name of the next selected shape is spoken to you, although with default settings both are rectangles and you will only note which one is selected from the changes that you hear when you move the shape around with your arrow keys. The up and down arrow keys give corresponding pitch changes as the selected shape moves up and down, while the left and right arrow keys change the timing and stereo position of the shape's sound as this changes the horizontal position of the shape relative to any other shapes in the soundscape. This description may sound more complicated then it really is, so just try it and you'll soon find out. As stated before, instead of two filled rectangles, you can select many other combinations of shapes via the menu Edit | Exercise Preferences | Randomly Placed Shapes. You will need your screen reader to navigate this dialog. You can set the number of rectangles, circles, triangles and line segments. So by default you have 2 rectangles, 0 circles, 0 triangles and 0 line segments. If you are new to vision, hearing two shapes in one soundscape scan can still be a bit confusing, and it may then be advisable to first go back to sounding only one shape, and move that shape around with your arrow keys. Try that for various shapes of choice until you feel comfortable in understanding what you hear in the soundscapes. Subsequently change settings to get two shapes in a soundscape scan, and try telling which is which.

  Another built-in exercise mode for practising the interpretation of soundscapes is the game of Tic-Tac-Toe activated by Shift F11.

• To offer a better trade-off between visual detail and peripheral sight, The vOICe offers a foveal mapping option (toggled by Shift Alt F4) that enlarges the central part of the view. In combination with a wide-angle lens this preserves the peripheral view for detecting obstacles and events in the visual periphery, while offering a detailed central view.
  

  A settings menu (toggled by Shift Alt 4) enables correction for a camera view that is often physically somewhat misaligned with the frame of camera glasses. It lets you adjust the effective viewing direction through software. The vOICe's foveal mapping has the additional property that horizontal and vertical straight lines remain perfectly straight, thus limiting the inevitable distortion associated with a foveal mapping. Unlike with for instance barrel distortion, an upright rectangle remains a rectangle with straight edges no matter if the rectangle appears in the foveal area, in the periphery, or in both.

• The possibility to record an AVI clip (digital video plus synthesized soundscapes) of any interesting landmark, route or environment.

  

  You need not worry if sounds appear to hiccup while saving the .avi file. This is a side-effect of the large CPU load, but it should normally not affect the sounds as stored in the .avi file: you should hear good results upon playback.

• The possibility to save a video frame and/or save a soundscape to disk.

  

  Individual video frames are stored in .bmp format, while individual soundscapes are stored in .wav format. Video frames can also be copied directly to the clipboard. A saved BMP image file may be imported into a suitable Braille printer or embosser to generate tactile graphics corresponding to the video image, for instance to provide tactile reference material for training purposes. You can even e-mail a live video snapshot and its soundscape directly via the integrated e-mail support of The vOICe.

  Optionally, you can also install extension modules for MP3 and JPEG support:

  

  The MP3 add-on option can be used for generating MP3 sound files, which are much smaller than WAV sound files. The JPEG add-on option is available for saving JPEG image files, which are generally much smaller than BMP image files.

  

• The possibility to browse the web for images and hear their soundscapes, using The vOICe Internet Sonification Browser dialog, which pops up when pressing Control U. You can provide the URL of an image to hear that image directly, or provide the URL of a web page, which will then be parsed to give a list of images and links in that page. The listed images and links can be activated to give either a soundscape in case of an image or a new list in case of a web page link.

• The possibility to restore the original audio settings, by pressing Shift F1 or Control F1. Most program settings, including the soundscape preferences, are saved and restored across program sessions, so this option allows you to restore the recommended built-in ``factory'' defaults. Especially when you get confused about the program settings you are using, it can be useful to switch back to the original settings. Two sets of built-in defaults are available: pressing Shift F1 restores the "best" audio settings, while Control F1 restores good non-spatialized ("dry") audio settings, resulting in less CPU load than when using Shift F1.

• The possibility of mobile OCR (optical character recognition) through a Mobile OCR extension with an external OCR engine. Results of mobile or portable OCR will strongly depend on the recognition accuracy of the OCR engine for extracting text from live video.

• Interfacing support for eye-tracking and other third-party applications. The vOICe auditory display view may be controlled via a head-mounted eye-tracking system (which can be optical or EMG-, EEG- or EOG-based), thus allowing for the auditory display equivalent of visual saccades.

In the table below, an overview is given of the main function keys and a number of keyboard shortcuts.

 

Screen readers for Microsoft Windows
	JAWS for Windows	tested OK
	Thunder	tested OK
	eSpeak	not tested
	NVDA	not tested
	Hal	not tested
Advantage of Windows 2000 and XP or later
Audio conflicts resulting in "audio hardware in use"  type of error messages should no longer occur when using Windows-2000 or Windows XP and later, which include a generic low-level audio mixer for all sound cards. These audio conflicts should also not occur when using The vOICe Learning Edition with the Creative Labs SoundBlaster Live! sound card, which is capable of mixing multiple sound streams at once. This works for users of Windows-95 and later. Users of Windows-98 may check if their sound card has a WDM driver (WDM: Win32 Driver Model), which is Microsoft's software solution for the infamous audio conflicts. The WDM driver for your sound card also transparently mixes audio streams from different applications within the Kernel Mixer (KMixer). Note that WDM drivers are not available for Windows-95, and that not all sound cards for Windows-98 come with a WDM driver. If none of the above solutions are available to you, the only remaining workarounds are the Control F2 key combination to release the audio device or, alternatively, the use of multiple soundcards: The vOICe for Windows allows you to select a specific sound card to use for the soundscapes, so you could use one sound card for your screen reader and another sound card for The vOICe for Windows.

The vOICe for Windows runs on computers with Microsoft Windows. For general usage, you will then probably need one of the screen readers developed specifically to provide Microsoft Windows accessibility to the blind. To help you, links to web sites offering various screen readers with speech output are given in the table on the right. Note that under Windows-95 you may have to disable the audio output from your screen reader program before starting The vOICe Learning Edition, because The vOICe for Windows may otherwise be unable to lock and use your audio output device: under Windows-95 only one application can give audio output at any given moment. However, Windows-95 screen readers and other audio applications can be made compatible with The vOICe: an interfacing solution is provided to third-party software developers to make their Windows-95 audio applications compatible with The vOICe for Windows (as well as to gain software support for controlling the auditory display with head-mounted eye-tracking systems). Also, pressing Control F2 in The vOICe will release the audio device for use by any other audio applications such as screen readers or .wav sound players.

Teaching and research options: Some special options are meant mainly for use by sighted vision teachers involved in developing a soundscape interpretation training program, or for use by people preparing a conference demonstration, or for use by investigators studying the limits of human perception under controlled conditions. The vOICe for Windows therefore includes for you:

• The possibility to read images from disk and convert these existing image files into soundscapes. One or more images, as stored on disk, will then replace the direct video input from the camera. The images and corresponding soundscapes will be cycled through until a new image selection is cancelled to return to normal video processing. Multiple images can be selected by the sighted with the Control-key in combination with mouse clicks. However, keyboard-only control is also possible by tabbing the selection dialog until the file list gets focus, and then keeping the Control-key down while using the arrow keys to move among the files and pressing the space bar to select or unselect image files.

  Images stored on disk are insensitive to momentary lighting conditions or camera quality (video noise), they can be artificially created or edited, and they may thus be used to fully control and demonstrate various subtleties in soundscapes and soundscape perception. The example image shown on the left is one of the most famous standard images used in image processing research, known as the "Lena" or "Lenna" image. You can download this standard image in 256 × 256 pixel 256 grey-tone .bmp format for use with The vOICe for Windows by clicking the image on the left. Of course one can apply all kinds of image processing to this and other images and then load the processed variants into The vOICe for Windows in order to further explore the effects of various image enhancement algorithms on auditory perception.

• The possibility to interpret BMP images using third-party software. While The vOICe for Windows is running, you can through a single key combination (Control R) order it to save the current video frame in .bmp format, run a batch file and resume normal processing when the batch file has finished. From within the batch file recognize.bat you can call any set of programs to process the video frame. For instance, specialized software could be used to recognize printed words and short texts through optical character recognition (OCR) applied to the vOICe.bmp image file. The result could then be sent to a speech synthesizer, and when it is done and the end of the batch file is reached, normal soundscape generation by The vOICe will resume. This simple and flexible open interface enables third-party software developers to add their own dedicated tools to the general vision substitution and synthetic vision framework of The vOICe, thus making the recognition of certain specific classes of visual patterns easier for the user. The vOICe's general mapping already gives you the possibility to learn to understand any visual pattern directly from the soundscapes, but of course it is much easier to understand text recognized via OCR and spoken via speech synthesis than it is to first learn to understand the complicated soundscapes of printed text. Apart from the automatic recognition of text through OCR, one can also think of reading and interpreting bar codes, currency, signs, or logo's, automatic face recognition, object recognition, etc.