HU, French research: Sounds can help blind ‘see’
Researchers use non-invasive sensory aids to provide visual information to the blind using their existing senses.
FUNCTIONAL MAGNETIC resonance imaging. Photo: HUJI.ac.il
The blind can’t see with their eyes, but they can see with their “mind’s eye” if
they are able to activate it through sounds and sensory substitution devices
(SSDs). Researchers at the Hebrew University of Jerusalem and colleagues in
France have just published an article on this in the prestigious neuroscience
Common wisdom has it that if the visual cortex in the
brain is deprived of visual information in infancy, it may never develop its
functional specialization properly, making sight restoration later in life
But researchers have found that congenitally blind
people who use specialized photographic and sound equipment can actually “see”
and describe objects and even identify letters and words.
The study by
Prof. Amir Amedi of HU’s Edmond and Lily Safra Center for Brain Sciences and the
Institute for Medical Research Israel-Canada, and doctoral candidate Ella
Striem-Amit, demonstrated how this achievement is possible through the use of a
unique training paradigm using SSDs.
The article, titled “Reading with
Sounds: Sensory Substitution Selectively Activates the Visual Word Form Area in
the Blind,” was written with help from Prof. Laurent Cohen and Prof. Stanislas
Dehaene of Pierre and Marie Curie University’s faculty of medicine, INSERM and
the College of France in Paris.
These are non-invasive sensory aids that
provide visual information to the blind via their existing senses. For example,
with a visual-to-auditory SSD in a clinical, or everyday setting, users wear a
miniature camera – connected to a small computer or smartphone – and stereo
The images are converted into “soundscapes,” using a
predictable algorithm, allowing the user to listen to and then interpret the
visual information coming from the camera.
The blind participants using
this device reach a level of visual acuity technically surpassing the criterion
set by the World Health Organization for blindness – as published in a previous
study by the same group.
The study shows that following – a dedicated but
relatively brief – 70 hours of unique training paradigm, developed in the Amedi
lab, blind people could easily use SSDs to characterize images into object
categories, such as of faces, houses, body shapes, everyday objects and
They could also identify even more complex everyday objects –
locating people’s positions, identifying facial expressions, and even reading
letters and words.
The study went on further to actually test what
happens in the brain when the blind learn to see with
Specifically, the group tested the ability of this high-acuity
vision to activate the supposedly dormant visual cortex of the blind, even
though it was taught to process the visual images through sounds only in
The team used functional magnetic resonance imaging to measure
the neural activity of people blind from birth as they “saw” – using the SSD –
high-resolution images of letters, faces, houses, everyday objects and
Surprisingly, not only was their visual cortex activated by
the sounds, their brain showed selectivity for visual categories which
characterize the normally developing, sighted brain.
A specific part of
the brain, known as the Visual Word Form Area (VWFA) that was first discovered
in sighted people by Cohen and Dehaene, is normally very selective.
sighted people, it has a role in reading, and is activated by seeing and reading
letters, more than by any other visual object category. Surprisingly, the same
was found in this area in people deprived of sight. Their VWFA, after only
dozens of hours of training in SSD use, showed more activation for letters than
for any of the other visual categories tested.
In fact, the researchers
found, the VWFA was so plastic to change, that it showed increased activation
for SSD letters after less than two hours of training by one of the study
“The adult brain is more flexible that we thought,” said
Amedi. In fact, this and other recent research from various groups have
demonstrated that multiple brain areas are not specific to their input sense
(vision, audition or touch), but rather to the task – or computation – they
perform, which may be computed with various modalities.
All of this
suggests that in the blind, brain areas might potentially be “awakened” to
processing visual properties and tasks even after years, or maybe even lifelong
blindness, if the proper technologies and training approaches are used, said
The findings also give hope that reintroduced input into the
visual centers of the blind brain could potentially restore vision, and that
SSDs might be useful for visual rehabilitation.
“SSDs might help blind or
visually-impaired individuals learn to process complex images, as done in this
study, or they might be used as sensory interpreters that provide
high-resolution, supportive, synchronous input to a visual signal arriving from
an external device such as bionic eyes,” explained Amedi.
sight, though not conventional in that it does not involve activation of the
ophthalmological system of the body, is no less visual in the sense that it
actually activates the visual identification network in the brain, the
After the training program, participants could assign
soundscapes to their visual categories and also determine multiple features of
the stimulus (such as hairstyle in a face image, number of floors and windows in
a house image, and body posture in a body-shape image), enabling them to
differentiate between objects within categories.