The vOICe Brainwave Entrainment

:-( Warning: this is still a highly experimental feature!

[Available upon registration of The vOICe for Windows]


« The vOICe Home Page
« The vOICe for Windows

One of the ultimate goals of sensory substitution is to offer not only the information but also the qualia (sensations) of a missing sense via another sense. The vOICe technology for the blind thus aims to offer the qualia of light perception and "true" (but low) vision in general via live camera input encoded in sound. Prior research suggests that combinations of brainwave frequencies and phase synchronization and desynchronization are associated with conscious perception and attentional selection, multisensory processes, sensory binding and sensory integration. In particular, gamma-band activity (GBA) and theta waves appear to correlate with these processes and thus with what is often called the "binding problem". The vOICe technology now supports brainwave entrainment (brainwave synchronization) through externally applied auditory stimuli that include a superposition of multiple independent binaural beats and monaural beats in typical δ, θ, α β and γ brainwave frequency ranges. Whether or not this will contribute to crossmodal binding as needed for truly visual experiences through sound remains to be investigated.

Registered users can invoke The vOICe entrainment preferences dialog via the Edit menu or by pressing Control B. Brain waves may after some period of use lock to monaural and/or binaural beats. Typical brainwave bins include the delta range (<4 Hz), theta (4-8 Hz), alpha (8-12 Hz), beta (12-30 Hz) and gamma (>30 Hz). Note that there exists no universal standard for these ranges,
Breaking the qualia barriers...
Ingredients meant to break the qualia barriers as needed for truly seeing with sound without resorting to drugs:

  • Encode and preserve visual view content in sound by using The vOICe, preferably using an immersive fully mobile setup for active sensing.

  • Apply mental imagery to visualize the soundscape content.

    Mindfold plus headphones, or Darth Vader?

  • Apply complete visual deprivation (light deprivation) through quality blindfolds to increase short-term neural plasticity.

  • Gesture (parts of) the soundscape content while listening, or clap your hands in sync with soundscape content, and make use of active movement and tactile feedback on view content wherever applicable and feasible. Use your hands, or "draw" soundscape content with the tip of your tongue against your palate.

  • Apply The vOICe brainwave entrainment to facilitate long range coherence between auditory (input) and visual (rendering, perception) areas in the brain.

  • Let the brain adapt through extensive training.

so definitions may vary a bit with different literature references. Here we used a representative range set. Coherent high-frequency oscillations, particularly those around the "40 Hz" gamma frequency, are often reported in EEG or MEG measurements of brainwave activity for multisensory or cross-modal perception, possibly involving long-range thalamocortical oscillations or purely cortical oscillations. Long range gamma-band synchrony seems to be further associated with the emergence of coherent conscious percepts. Other qualitative observations suggest that local synchronization in the brain evolves mostly in the gamma frequency range, medium range synchronization in the beta1 frequency range (12–18 Hz), and long range synchronization in the theta and alpha frequency range. The more local feedback loops thus seem to give higher frequencies. A possible interpretation is that the brain contains a number of somewhat independently running processing modules, each with its own characteristic oscillating frequencies, while the onset of cooperation among these modules as needed for a complex (e.g., multisensory) task may be arranged through temporary phase-relationships among the participating modules. In technical terms, these modules would then act much like coupled oscillators and PLLs (phase-locked loops) that can phase-lock to external stimuli, including stimuli coming from other PLLs. With phase-dependent modulation of signal transmission efficacy, phase-locking can provide a mechanism for on-the-fly allocation of multisensory processing resources. It should be noted that the brain's oscillatory responses to external oscillatory signals may be evoked (phase-locked) or induced (not phase-locked). The direction of information flow in coupled subsystems may be estimated from partial directed coherence (PDC, Granger causality) measures.

The purpose of The vOICe brainwave entrainment is thus to facilitate phase-locking of brain waves as needed for effective crossmodal synesthetic processing, by adding external auditory stimuli with frequencies close to natural brainwave frequencies involved in transient long-range phase-locking for sensory integration with conscious visual perception (i.e., with "visual awareness" or "visual consciousness" as philosophers may put it). It may also be viewed as a form of externally driven but non-invasive neuromodulation. The external auditory signal provides weak beats that may help induce sudden changes in the oscillatory states of the brain that correspond to qualitative state space transitions in brain activity and subjective perception. Signal transfer among distant brain modules may be dynamically switched by transitioning between anti-phase oscillations and near synchrony. Basically, our hypothesis is now that by offering a rich set of visual inputs (live camera images encoded on a non-visual carrier such as sound) plus binaural and monaural stimuli that are consistent with brainwave patterns observed during multisensory processing, normal visual perception and/or mental imagery, we will increase the probability of inducing conscious visual percepts, and together with gesturing soundscape content and adaptation through training to guide brain plasticity hopefully bring the spatial distribution and timing of brain activity above the critical thresholds as required for conscious and meaningful visual perception. This would likely correspond to a major reduction of the phosphene threshold (PT) as measured via transcranial magnetic stimulation (TMS). By analogy, people with blindsight can be viewed as having (much) elevated thresholds for conscious visual perception, while grapheme-color synesthetes may have lowered thresholds for crossmodal conscious visual perception, and augmented cognition and memorization skills may be associated with improved multi-modal sensory integration.

The brainwave entrainment feature is meant as a temporary training tool, which adds brainwave frequencies to try and induce or stimulate cross-modal communication from auditory brain regions to visual brain regions. After paving the way by strengthening these connections, the brainwave entrainment may no longer be needed for inducing visual percepts, and the standard modes of The vOICe can be used again for best view quality.

Training patterns with two rectangles and one line The vOICe entrainment preferences

The presets available in the preferences dialog are mostly to illustrate a few characteristic settings. Better settings may exist. Preset 6 is currently among the favourite settings (keyboard shortcut: Control B to get to the brainwave entrainment dialog, followed by Alt 6 Enter to activate Preset 6).

Warning: Beware that little is known about the short-term and long-term effects of brainwave entrainment, so one has to consider possible risks, even if these are somewhat theoretical. For instance, it is conceivable that brainwave entrainment could cause epileptic seizures in susceptible individuals (although - unlike photogenic seizures - audiogenic seizures are very rare in humans), and these people should not use the brainwave entrainment options except under proper supervision by or with consent from a licensed physician or other qualified health care professional. The same applies for individuals with (hallucinatory) mental problems such as schizophrenia, for those prone to developing psychosis, and possibly for a variety of other medical and mental conditions. Moreover, brainwave entrainment should never be used in mobile situations or when operating dangerous machinery, because it may induce changes in brain state that may include falling asleep. Other side effects may exist when using "agressive" settings, possibly including dizziness, loss of balance, nausea, hallucinations or mood changes. You may be irritable (emotionally unstable and explosive) and feel unusually tired at times as your brain is in a process of major adaptations. In other words, use any of this only at your own risk, and where applicable under proper guidance. Also, brainwave entrainment with The vOICe has not yet been proven effective for any purpose, may include placebo effects to be accounted for, and is therefore highly experimental if not hypothetical. A lot of questionable if not outrageous claims have been made regarding the effects of binaural beats in particular, often in "New Age" like contexts, and with so-called "i-dosing" or "digital drugs", so take care. Also, the scientific evidence for changing brainwave frequencies through binaural beats is still very meager at best. This web page will be updated accordingly as more information becomes available over time.

You can watch a free demonstration  video clip of The vOICe running in exercise mode with randomly placed shapes and soft beats. This particular video clip is based on preset 7 with theta and gamma frequency binaural beats and alpha frequency monaural beats. You may set your media player to autorepeat (loop) the clip.

Binaural beats (dichotic beats) occur with small frequency differences for left and right ear, for carrier frequencies up to about 1.5 kHz (low to medium pitch). This is likely related to the phase-locking limits of the human hearing system. With The vOICe you may define up to 4 independent binaural beat frequencies. Their carriers lie interleaved in the soundscape spectrum, using all (default) 64 frequency channels of The vOICe. Beat frequencies may be specified up to 100 Hz (each specified interaural frequency difference, IFD <= 100 Hz). You may also adapt the scantime T in order to fit an integer number of (slow) waves within the period T while maintaining integer ratios among different binaural beat frequencies. The vOICe sets the initial binaural beat phases by default to zero.

For instance, to combine a 40 Hz gamma-range beat (gamma rhythm) with a beat in the theta-range that runs a factor seven slower (at 40/7 = 5.71429 Hz), you can take into account that 6 periods of this slower wave fit in exactly 6 * 7/40 = 1.05 seconds. The divisor 7 has been suggested in relation to short-term memory capacity, typically running up to about seven items for most people.

Input frequency Hz;  divisor        suggested T= s, such that periods of frequency Hz fit in time T.

Similarly, with a divisor 6, a 40 Hz gamma-range beat combines with a 6.66666 Hz theta beat (40/6), for which exactly 7 periods fit in 7 * 6/40 = 1.05 seconds. Another example is to combine a 40 Hz gamma-range beat with a beat in the beta1-range (12–18 Hz), where an integer divisor of 3 would apply (40/3 = 13.33333 Hz), such that 14 periods of the slower wave fit in exactly 14 * 3/40 = 1.05 seconds (again!). The beta1 frequency range has been reported in synchronization between temporal and parietal cortex during multimodal object processing (von Stein et al.)

Use of stereo headphones is required to properly perceive binaural beats.

Monaural beats can occur when frequencies are an offset plus an integer multiple of a lower (beat) frequency, causing perception of the physically missing fundamental. With The vOICe you may define up to 2 monaural beat frequencies. Their carriers lie interleaved in the soundscape spectrum. Monaural beat frequencies of up to 20 Hz are allowed. The vOICe sets the initial monaural beat phases to random values. Amplitude modulation - as with the sound bursts in so-called isochronic tones, or when using so-called transposed tones - for generating perceivable beats for brainwave entrainment is generally not advised for regular use of The vOICe, because a superimposed amplitude modulation (and also the common use of click trains) would interfere with the visual semantics of The vOICe image to sound mapping: both simple amplitude modulation and click trains would correspond to vertical stripe artifacts. On the other hand, amplitude modulation may support brainwave entrainment as such. Therefore, although it may serve enhanced training, isochronic tones, clicks and transposed stimuli should under normal conditions only arise from actual visual content with vertical stripes. After all, entraining the brain of blind users with actual visual content remains our primary goal!

The table below outlines some example settings that you could try:

skip table
Binaural beat settings Monaural beat settings T Comments
beat 1 (Hz) beat 2 (Hz) beat 3 (Hz) beat 4 (Hz) beat 1 (Hz) beat 2 (Hz) (s) divisors relative to 40 Hz
5.71429 40 40 40 5.71429 5.71429 1.05 θ and γ, divisor 7
20 20 40 40 - - 1.00 β and γ, divisor 2
13.3333 13.3333 40 40 5.71429 5.71429 1.05 β, γ and θ, divisors 3 & 7
10 10 40 40 5 5 1.00 α, γ and θ, divisors 4 & 8
5 5 40 40 10 10 1.00 α, γ and θ, divisors 4 & 8
6.66667 6.66667 40 40 6.66667 6.66667 1.05 θ and γ, divisor 6
5.71429 5.71429 40 40 5.71429 5.71429 1.05 θ and γ, divisor 7
5.71429 5.71429 6.66667 6.66667 11.42857 11.42857 1.05 θ and α
5.71429 5.71429 40 40 11.42857 11.42857 1.05 θ, γ and α, divisor 7, ×2
5.71429 5.71429 40 40 17.14286 17.14286 1.05 θ, γ and β, divisor 7, ×3
5.71429 40 6.66667 40 5.71429 6.66667 1.05 θ and γ, divisors 6 & 7
5.71429 6.66667 20 40 5.71429 6.66667 1.05 θ, β and γ, divisors 2, 6 & 7

 
One currently envisioned experimental training usage of brainwave entrainment is to combine it with the free-running exercise mode (toggled by function key F11) at night, while lying in bed in near-total darkness before going to sleep. In the exercise mode, the normally black background can now be made dark grey using the background slider. This option is meant to always have at least some weak background rhythm even if there are no or only very small shapes. Thus you may focus attention on the beats, on the shapes in the soundscapes, and on any visual effects that may include spontaneous vivid imagery and the normal hypnagogic hallucinations that people have before falling asleep. Doing this for half an hour up to an hour daily may within a few weeks or months of practice give "results", but what the typical results will be remains to be investigated. Side effects and after effects may include having a different quality of sleep (such as unusually deep sleep), irritability, fatigue, increased sensitivity to sound and light, and associated (mild) headaches. This may be interpreted more positivily as indicating that your brain is in a process of adaptation, much like a new fitness program may initially cause physical discomfort in the form of muscle pain. You may also start having occasional dreams about seeing sound, just like you would likely have with any novel and attention demanding activity. In any case, brainwave entrainment is for most people not expected to give any immediate spectacular results because the brain first needs to get better at giving a frequency-following response and master phase-locking to external stimuli, so you need some patience, as with any kind of training. Ideally, any hypnagogic hallucinations or phosphenes would in due course begin to match the soundscape content, effectively resulting in cross-modal sensory binding with conscious visual percepts. Different settings for the exercise mode can be configured via the Edit | Exercise Preferences | Randomly Placed Shapes menu, for instance to challenge visualization by adding a random line or another shape to the randomly placed rectangles. However, for easy visualization it is recommended to first work with a view that contains just one line and one rectangle such that your attention can be focussed on obtaining and guiding any visual percepts. Later on you can select views with, say, two lines, two rectangles and one or two circles. Adjust the sound volume to a comfortable level. During initial training you should probably welcome and not attempt to suppress any remote resemblance between sounds and visual effects, even if for instance the visual effects occur in the "wrong position" as compared to the sounding shapes, for instance in the periphery of your view, or in the wrong orientation. After all, the brain has multiple sensory subsystems, and one cannot reasonably expect them to create perfect matches right from the start. You may for instance experience a modulation of residual background light over your full field of view, appearing as weak background flashes that run more or less synchronized with the theta beats (if these were selected in the beat settings), or you may see fuzzy 1D wire grids. You may also experience visual effects at the "wrong time", in the form of visual flashbacks (mental images) of earlier views used during training, and at moments that you are not even using The vOICe.

Beware, brainwave entrainment will likely not give you instantaneous visual percepts, and it is also not a replacement for prolonged regular training and immersive use of The vOICe. By acting as an oscillatory "scaffold" for guiding and sculpting neural circuit dynamics and functional connectivity, it is one of multiple ingredients that together should lead to desired results by helping the brain move closer to crossing the thresholds for conscious visual perception through sound. Currently there is no evidence that this supplemental approach works as intended, which is why the approach counts as highly experimental. For example, in a 2012 paper by Vernon et al.,  ``Tracking EEG changes in response to alpha and beta binaural beats,'' no measurable effect of 10 Hz alpha or 20 Hz beta binaural beats was found.

Training with brainwave entrainment is meant only as a stepping stone for learning to see with sound, and with more experience the brain should be able to link auditory processing to visual processing and visual experiences without the need for any artificially added brainwave frequencies to induce neural synchrony for sensory integration.

For related topics, you can visit the web pages on sensory substitution, artificial synesthesia, mental imagery and training. The latter page includes a discussion of the possible supplementary role of physical action through gesturing soundscape content. Of tangential interest is also the work on learning in dynamic neural networks, which can be applied for efficient modelling and simulation of coupled oscillator circuits, by already including basic oscillatory behaviour in single artificial neurons that each contain a second order linear ODE (ordinary differential equation).

Chances of obtaining visual percepts may be further enhanced through a phenomenon called stochastic resonance: The vOICe generates patterned noise that encodes the actual visual content of the view to be perceived, but even auditory noise itself can via cross-modal stochastic resonance help bring very weak (hallucinatory or real) visual signals above the thresholds for conscious detection (Cf. Manjarrez et al., 2007, who suggest that auditory noise could be employed in vision rehabilitation in order to improve the detection of weak visual signals, and Dylov and Fleischer, 2010, who applied nonlinear coupling to grow signal from noise in dynamical stochastic resonance).

Stochastic resonance: detection threshold crossed due to added noise

In general, cognition and perception are in part probabilistic in nature, and statistical approaches based on stochastic resonance and Bayesian modeling may lead to practical probabilistic models of (development of) cognition and perception, multisensory integration and sensorimotor integration. Clearly, one would hope that the human brain can learn to act as a so-called "optimal observer" for soundscapes from The vOICe. Finally, the partial thalamic stimulation by auditory input (e.g., from brain stem via thalamus to auditory cortex) may further contribute to sensory binding and other large-scale integrative processes in the brain, but it is still unclear whether this effect is of practical significance.


Selected related or tangential literature on brainwave coherence, visual qualia and entrainment:

Recent publications about brain_waves and binaural beats

Selected further resources:
 www.40hz.net (Andreas Engel, University Medical Center Hamburg-Eppendorf, Germany)
 www.danielsenkowski.com (Daniel Senkowski, University of Hamburg, Germany)
 www.markussiegel.net (Markus Siegel, University of Tübingen, Germany)
 www.klab.caltech.edu (Christof Koch, California Institute of Technology, USA)
 www.scholarpedia.org/article/Binding_by_Synchrony (Wolf Singer, Max Planck Institute for Brain Research, Germany)

Note: Any therapeutical uses of The vOICe brainwave entrainment, for instance as part of experimental treatments of auditory or cross-modal gating deficits in schizophrenia, should only be tried under proper medical supervision by qualified healthcare professionals. The vOICe technology is intended for educational uses, to stimulate academic and clinical research, to learn about vision and to enable training for visual skills and experiences. It is not intended to diagnose, treat, cure, or prevent any disease or condition, and has not been proven effective or safe for any such purposes.

Copyright © 1996 - 2024 Peter B.L. Meijer