Researchers at UC San Francisco have developed a "speech neuroprothesis" that translates brain signals into text on a screen, and have successfully installed it on a man with severe paralysis.
The achievement is a result of more than a decade of effort by UCSF neurosurgeon Edward Chang MD and was developed in collaboration with the first participant in a clinical trial. The study appeared in New England Journal of Medicine.
“To our knowledge, this is the first successful demonstration of direct decoding of full words from the brain activity of someone who is paralyzed and cannot speak,” said Chang. “It shows strong promise to restore communication by tapping into the brain's natural speech machinery.”
The hope is that the speech neuroprothesis will return the ability of communication to thousands of people a day, who lose their speech to stroke, accidents or diseases.
Chang's study differs from previous work on the subject because past methods have worked based on a spelling-based approach, where the communication was spelled out letter by letter. Chang's approach is more compatible with the fluid nature of actual speech, thus allowing for more rapid and organic speech.
“With speech, we normally communicate information at a very high rate, up to 150 or 200 words per minute,” he said, noting that spelling-based modes of communication work at a slower rate. “Going straight to words, as we’re doing here, has great advantages because it’s closer to how we normally speak.”
Chang's progress was done in collaboration with patients with normal speech ability at the UCSF Epilepsy Center. Their brains were being scanned, using electrodes placed on the surface of the brains to locate the source of their seizures, and they volunteered to have their scans analyzed for speech-related activity. The success of the study is largely due to the success of these analyses.
Chang and his colleagues developed a system that looked promising, but they couldn't know if it would work until they tried it on someone with paralysis.
In order to test if this would work, Chang partnered up with Karunesh Ganguly MD, PhD, an associate professor of neurology, to create the Brain-Computer Interface Restoration of Arm and Voice (BRAVO) program.
The first patient to take part in the program was a 30-year-old man, who had suffered a stroke 15 years ago. Among other areas of his body that were paralyzed, the connection between his brain and his vocal cords was damaged.
The patient, who asked to be referred to as BRAVO1, worked with researchers to build a vocabulary of 50 words with which he could build hundreds of sentences.
For the study, Chang implanted an array of electrodes on BRAVO1's brain. After he recovered, the team recorded 22 hours worth of neural activity in which BRAVO1 attempted to say each of the 50 words many times.
When the neural scans were ready, the other two lead authors of the study, Sean Metzger, MS and Jessie Liu, BS, both bioengineering doctoral students in the Chang Lab used custom neural network models to translate the signals into words. Thus, when BRAVO1 tried to speak, the network was able to read the signal and turn it into words.
The team then built short sentences and had BRAVO1 try to say them again and again until the words appeared on the screen.
They then began asking him simple questions, so that he would try and say unscripted sentences, and that also worked.
The system was found to be able to identify 18 words per minute at a 93% accuracy rate.
for the future, the team intends to expand the trial to include more participants. Currently, the study focuses on one participant and has a limited vocabulary. All the same, it is considered a great success and further development is expected.