Shepard-Metzler Mental Rotation using VRML/X3D

After you have installed a VRML 2.0 or X3D compliant browser, and using the screen sonification procedure as explained in full detail on the Accessible VRML/X3D for the Blind page, you can try and listen to the soundscapes of interactive 3-dimensional VRML/X3D views of Shepard-Metzler objects. These objects consist of ten cubes strung together face-to-face into an arm-like structure, having exactly three right-angled "kinks" or "elbows". Such objects were used by Shepard and Metzler in their classic 1971 Science paper to show that the time that sighted people need to mentally rotate a visual view of a 3D object depends linearly on the rotation angle for matching two views on an object: the larger the angular difference, the more time sighted people need to mentally rotate and align objects.

Note: if you are blind and the above verbal description of the object shapes sounded unclear to you, think of the close analogy of a piece of metal wire bent at right angles in three places. Here it is much the same but with ten cubes stitched together to make for a kind of "fat" rectangular wire.

Now it will be an extremely demanding task to learn and make sense of the soundscape counterparts of the Shepard-Metzler shapes, but at least you will notice the changes in auditory appearance as the objects are rotated while listening to them via full screen sonification. For the examples on this page, the arrow keys for the VRML/X3D scene have been programmed to let you turn the 3D object around to view and hear it from all sides. Just click the Shepard-Metzler shepard.wrl/shepard.x3d link to enter the VRML/X3D world via your browser. Full screen sonification with The vOICe is activated by pressing Shift F9 while The vOICe is running and has focus.

If desired, you can apply Control arrow up and Control arrow down to zoom in and out, while the arrow keys in combination with the Alt key let you pan.

Slightly differently shaped Shepard-Metzler 3D VRML/X3D objects are available via the Shepard-Metzler 1 links shepard1.wrl and shepard1.x3d and Shepard-Metzler 2 links shepard2.wrl and shepard2.x3d.

 

The basic approach as outlined on this page may also find applications in scientific experiments with cross-modal binding and sensory integration in sighted subjects (weak audio-visual crossmodal binding has been suggested for people with autism).

Related literature:

B. Alivisatos and M. Petrides, ``Functional activation of the human brain during mental rotation,'' Neuropsychologia, Vol. 35, pp. 111-118, 1997.

M. S. Cohen, S. M. Kosslyn, H. C. Breiter, G. J. DiGirolamo, W. L. Thompson, A. K. Anderson, S. Y. Bookheimer, B. R. Rosen and J. W. Belliveau, ``Changes in Cortical Activity During Mental Rotation: A mapping study using functional magnetic resonance imaging (fMRI),'' Brain, Vol. 119, pp. 89-100, 1996.

C. Garbis, ``Spatial Representation and Haptic Mental Rotation,'' Lund University Cognitive Studies, LUCS 56, 1997.

A. Jackson, ``The World of Blind Mathematicians,'' Notices of the American Mathematical Society, Vol. 49, No. 10, 2002. Available  online (PDF file).

N. H. Kerr, “The Role of Vision in “Visual Imagery” Experiments: Evidence from the Congenitally Blind”, Journal of Experimental Psychology: General, Vol. 112, pp. 265–277, 1983.

F. Rösler, M. Heil and E. Henninghausen, ``Topographic differences of slow event-related brain potentials in blind and sighted adult human subjects during haptic mental rotation,'' Cognitive Brain Research, vol. 1, 145-159, 1993. R. N. Shepard and J. Metzler, ``Mental rotation of three-dimensional objects,'' Science, Vol. 171, pp. 701-703, 1971.

S. Ungar, M. Blades and C. Spencer, ''Mental Rotation of a Tactile Layout by Young Visually Impaired Children,'' Perception, Vol. 24, pp. 891–900, 1995.

J. Zimler and J.M. Keenan, ``Imagery in the congenitally blind: how visual are visual images?'' Journal of Experimental Psychology: Learning Memory and Cognition, Vol. 8, pp. 269-282, 1983.