Accessible VRML/X3D for the Blind
Safe orientation and mobility training in 3D virtual environments!
Experience for yourself visual perspective, occlusion, and more!
The vOICe technology even supports access to 3D game worlds
« The vOICe Home Page
« The vOICe for Windows
This web page is obsolete and will no longer be maintained.
VRML and its successor X3D are languages for building virtual 3-dimensional environments,
often called "worlds". Sighted people can navigate these worlds using a VRML or X3D browser.
Until now, purely visual virtual worlds have been inaccessible to blind people.
This has changed! When using the screen sonification option of
The vOICe for Windows seeing-with-sound software
in combination with a VRML/X3D browser, totally blind people can now independently
experience and navigate VRML/X3D scenes. This page explains how yet another
accessibility barrier has been taken down. An example is included of a purely
visual VRML/X3D environment that you as a blind visitor can (learn to) navigate.
Once again this will demonstrate how The vOICe technology makes vision accessible,
through audio rendering of arbitrary images for sound-controlled synthetic vision.
Moreover, with The vOICe auditory display approach, no expensive special hardware
nor changes to mainstream browsers are needed for access to virtual worlds:
It is Now available to you. Free
of charge!
Several MP3 audio samples on this page further illustrate the principles, but it
is highly recommended to install the required software and explore interactively
how the soundscape scenery around you changes as you move through a virtual
environment. There is no experience like an immersive experience. Try it!
Later on, similar approaches for VRML/X3D sonification may also become available
for mobile phones, for possible use in conjunction with
The vOICe MIDlet and The vOICe for Android.
Virtual reality mobility training
In order to view the VRML/X3D world available on this page, you first need to
install a VRML 2.0 or
X3D compliant viewer plugin
such as
Flux Player
from Media Machines
- unless you already have that.
Note that Flux Player requires clicking inside the Flux Player window (best done when maximimized)
to make it responsive to the arrow keys for turning a scene or object: normal tabbing or Alt-tabbing
to bring the player application or plug-in into focus does not suffice.
Secondly, you need The vOICe for Windows
software. Once you have all that, you can proceed
with the following recipe for hearing VRML/X3D worlds such as the one given
on this page. First read and perhaps copy to another document all steps
of the procedure such that you will remember or can find what to do after
leaving this web page to enter the VRML/X3D world. The order of the steps
may be important to avoid error messages on audio conlicts.
Procedure
- If necessary, make sure that you have properly installed a VRML/X3D
viewer plug-in, and that your Internet browser has loaded this web page.
- Start The vOICe for Windows (Control Alt v), and press the function
key combination Shift F9 to enter full screen sonification.
You should now be hearing whatever is displayed on your screen. A small
area of your screen will still be covered by The vOICe window.
- Minimize (Alt space n) The vOICe for Windows window.
This is needed to prevent that The vOICe window occludes part of your
browser window with the VRML/X3D scene, because normally The vOICe window
stays in front of all windows. Later on, Control Alt v can be used to
restore the normal window state for The vOICe to access its various
controls.
- Put focus back to your browser (Alt Tab) and maximize your browser
window (Alt space x) to make it occupy the entire screen and bring it in
front of any other windows.
This will give you the best view on the VRML/X3D world by using most of
the screen real estate. Later on, Alt Tab r can be used to restore the
previous non-maximized window size setting.
You may wish to disable the toolbars and status bar of your browser
window to further maximize your view on the VRML/X3D scene. These options
are typically set via the menus of your browser.
What about 3D Games for the Blind?
Technical feasibility of accessing non-VRML/X3D 3D game environments
|
Non-VRML/X3D gaming:
Using largely the same procedure, you can also access and hear the visual scenery in
most of the latest 3D games. Just start your favourite DirectX or OpenGL 3D graphics
game after applying steps 2 and 3.
Notes: Full accessibility or playability is not implied
or guaranteed. Some games will fail to cooperate with The vOICe sonification. Visual
contrast in 3D graphics games is often poor, resulting in low-contrast soundscapes.
Use of a multichannel soundcard or equivalent is recommended.
|
|
Click to hear The vOICe Half-Life MP3 soundscape (18K)
|
- Now enter the VRML/X3D world by activating The vOICe
VRML/X3D demo link.
- Test the basic operation of your navigation controls: keeping the
arrow-up key pressed, you will gradually walk forward. If you want to
move forward faster, use Shift arrow-up.
You should hear the rhythm of the pillars move closer, and next you will
move in between the two rows of pillars. If nothing seems to change in the
soundscapes, chances are that you need to force the focus of your browser
to the VRML/X3D view, because even if your browser has focus, focus within
the browser may be at some browser control other than the VRML/X3D scene.
Sighted users can click the left mouse button somewhere in the VRML/X3D scene
to put focus directly to the VRML/X3D scene, and blind users can best do
this with the browser window maximized in order not to "miss" the window
when clicking.)
An easy way to move to a number of predefined viewpoints is to press
the page down key a number of times. This will give you a useful
overview of the kinds of soundscapes and objects that you will likely
encounter when navigating around by yourself.
Blind users are welcome to report about their experiences on the
seeingwithsound user group.
Once all of this works as described, you can start navigating around the
VRML/X3D scene, using the arrow keys on your keyboard for full interactivity.
Arrow-up moves forward, and arrow-down moves backward. The shift key can be
used for more speed. The arrow-left key turns you several degrees to the left,
while the arrow-right key turns you several degrees to the right
Keeping the keys pressed will allow for more or less
continuous motion. Apart from these basic arrow key controls, some viewers may
support several other keyboard shortcuts that can be useful, and below we outline
a few keyboard controls of the older
Cosmo Player viewer.
For instance, pressing the delete or backspace key may undo your last movement.
Pressing Shift Home may fly you back to the initial entry viewpoint. Pressing the
page down key may automatically move to the next predefined viewpoint, if any,
while page up may return to the currently selected (predefined) viewpoint.
Alt arrow left and Alt arrow right may slide you sideways without changing the
viewing direction, and control arrow up and control arrow down may let your viewing
direction tilt up and down, respectively.
Again, all these extra features, as described for Cosmo Player, vary with the
type of viewer and may not be supported or may involve other keyboard shortcuts.
After the following brief description of The vOICe image to sound mapping,
we will continue with a more complete overview of the VRML/X3D scene, such
that you know what to look around for in this virtual reality training
example.
The vOICe's synthetic vision mapping sounds the VRML/X3D scene as displayed on
the screen by mapping brightness to loudness, elevation to pitch and lateral
position to time in each image scan (using stereo panning for enhanced perception).
Thus the greyscale content of any VRML/X3D scene can be rendered in sound.
If your sound card allows for multiple simultaneous audio streams, you will
not only hear The vOICe's soundscapes, but objects will also speak their names
whenever you touch them. Objects thereby double as their own talking signs.
As long as you do not touch or bump into objects, only the purely visual
content of the soundscapes will guide you through the visual environment.
This is the true equivalent of general vision, because it does not involve
any tagging of visual objects, and will work for all existing VRML/X3D worlds on
the web, including those that were not designed or adapted for accessibility!
The entry scene upon entering the VRML/X3D demo shows a view of a road
running towards the horizon. The grey road surface gives a soft
noise. The road has a wide dark grey verge, sounding as a still
softer noise. Visual perspective makes the road sound like a big
triangle with the point at the top: the so-called vanishing point
at the far end of the road.
Two fast rhythms correspond to two rows of pillars alongside the
road. There are ten pillars bordering on the left side of the road
and ten pillars bordering on the right side of the road, thus
forming two rows that run in the same forward direction as the
road. The dark grey road verge extends well beyond the left and
right of these rows of pillars, but the lighter grey road surface
does not. As you move forward (arrow-up or shift
arrow-up), you get closer to the two rows of pillars, making
them sound "bigger". Next, you will move in between the rows
of pillars, and if you continue, you will end up on an empty
stretch of road as the pillars move out of view behind you.
If you are currently just reading this web page, you can get some
idea of the soundscapes beforehand by listening to an MP3 audio
sample containing four soundscapes. If you have a media player
installed, try the four soundscapes
link to load the 64K MP3 audio sample, showing the VRML/X3D entry view,
a viewpoint closer to the pillars, and two viewpoints resulting from
turning some 30 degrees to the right while moving forward. In
the fourth soundscape, one pillar is particularly nearby. Neglect
the continuous high pitched sounds in the soundscapes: these just
sound the menu bar at the top side of the browser window. Similarly,
the lowest pitched sounds are from some VRML/X3D viewer controls
showing near the bottom of the screen. These sounds too should be
neglected because the visual controls are not part of the VRML/X3D scene.
Assuming again that you were actually navigating the scene,
having moved beyond the rows of pillars, you can turn around
with the left or right arrow keys (if supported by your viewer),
and if you do a 180 degree turn, you will hear the pillars again
that were behind you.
Moreover, there is a bright wall behind the pillars, sounding
as a band of rather loud noise. This wall was actually right
behind you when you first entered the scene, but you could
only have heard it by turning around: no eyes in the back of
your head.
Now with the pillars and the wall in view, move forward such that you
pass the pillars again, and listen for the gate in middle of the wall.
By the way, the wall runs perpendicular to the road. Readers of this
web page can try a 18K MP3 soundscape of a perpendicular
view on the wall with its gate, in which you can clearly hear the
interruption of the wall sound by the gate. Another 18K MP3 soundscape
shows a sideways view on the wall with the gate,
looking at the gate from a position somewhat to the right side of the
gate. The soundscape at the latter viewpoint is rather similar to the
demonstration sound as discussed on the "Bright
Wall with Gate" web page. Continue to head for the gate, adjusting
direction with left and right arrow keys as appropriate, and
moving forward with the arrow-up key. If you do this carefully,
you will not bump into the wall, but manage to move through the
gate. At that point, you will see a big sphere on your left and
a big cube on your right. These objects had been occluded by
the wall until you passed the gate to move to the other side
of the wall. You can head for the sphere or the cube, move
around them, turn around to see the wall from the other side,
and so on. You are entirely on your own now, but you cannot
be hurt! This is a major advantage of orientation and mobility
in virtual environments like these. Without risk, and without
special hardware, you can now learn all about visual perspective,
occlusion, parallax, shading, and so on and so forth. Orientation
and mobility experts and others are invited to build other VRML/X3D
worlds to arrive at a library of rich training environments, for instance
with various kinds of visual landmarks for learning to see with sound.
A totally different interactive VRML/X3D scene, showing the planet Saturn with its rings,
is given near the bottom of the planet Saturn page, while
mental rotation test objects are discussed on the Shepard-Metzler
page.
The same basic approach as outlined above for accessing VRML 2.0, VRML97 or
Web3D's X3D
can also be used with
Ajax3D (formerly at www.ajax3d.org),
Google
SketchUp,
Macromedia
Shockwave 3D
and Cycore
Cult3D,
as well as with scalable vector graphics (SVG, sample file molyneux.svg).
It can also be applied in multiplayer 3D worlds like
Second Life
(cf.
Eduverse
for a virtual education focus in the metaverse).
See also the interactive online 3D Maze example, and the
Blind drawing with Logo page for programming your own
2D and 3D shapes.
Other related projects include
Making VRML Accessible
at the Open Virtual Reality Testbed (OVRT) of the National Institute of Standards
and Technology (NIST).
For general information about VRML, visit the
Web3D Consortium website.
A publication on the use of visual-to-auditory sensory substitution by blind people
to access visual virtual environments is the paper
Perception of graphical virtual environments by blind users via sensory substitution" by Shachar Maidenbaum et al. in PLoS ONE 11(2), February 2016, e0147501.
Copyright © 1996 - 2024 Peter B.L. Meijer