Tag Archives: Miguel Nicolelis

Walking again with exoskeletons and brain-controlled, non-invasive muscle stimulation enabling people to walk

I have two news bits about paraplegics and the possibility of walking. The first is from Alberta, Canada and the second is from Brazil.

Alberta

The fellow in the video is wearing a robotic exoskeleton. As you can see, it’s not perfect but it represents an extraordinary breakthrough (from an April 16, 2019 article by Sarah Lawrynuik for the Canadian Broadcasting Corporation [CBC] Radio),

On his fifteenth birthday in December 2015, Calgary’s Alex McEwan was injured in a tobogganing accident with friends and lost the ability to walk. It’s the kind of change that could destroy a person, but Alex has thrived and is learning new skills. Watch him walk onstage, with some help from a powered exoskeleton, to receive his diploma. 1:21

Sometimes events conspire to move us in a completely unexpected ways. After his accident, Alex McEwan participated in a very special study (from an August 3, 2019 article by Colin Zak for Alberta Health Services),

Researchers at Foothills Medical Centre (FMC) are the first in Canada to examine the benefits of using an exoskeleton robotic device to rehabilitate patients with spinal cord injuries (SCI) in the days and weeks following their injury.

The device, known as the Ekso Bionic Exoskeleton, consists of a metal frame that supports and stabilizes a patient’s torso, core, legs and feet. It is moved robotically by a therapist, enabling patients with a spinal cord injury to get up and walk around. Although it is controlled by remote control, the device offers varying levels of physical control by the patient, depending on the nature and extent of their injury.

Dr. Ho [ Dr. Chester Ho, Head of Physical Medicine and Rehabilitation at FMC ] says exoskeletons may potentially promote recovery and reduce complications in SCI patients by reducing loss of bone and muscle mass caused by spending so much time lying down, and also improve breathing and bowel function.

The year-long study, which begins this summer, will include between five and 10 patients selected from across Calgary [Alberta]. It aims to examine whether treatment is safe and feasible in the days and weeks after an SCI. This study will be followed by larger studies involving more patients.

Participants in the study will receive 60-minute therapy sessions with the exoskeleton device two to three times a week, for a total of 25 training hours over an eight- to 10-week period. Safety and feasibility outcomes will be monitored and tracked by the research team throughout all sessions.

Before the advent of exoskeletons, rehabilitation for patients with an SCI required them to be hoisted with a physical therapist moving their legs.

“Every step is different with this device, so patients learn from their mistakes in real time. Patients really like to use the device; it gives them hope.”

Alex, [emphasis mine] 15, sustained a spinal cord injury while tobogganing last December.

He says rehabilitation sessions with the exoskeleton have made a difference in how he feels and gives him hope for the future.

Over 2 1/2 years later, the CBC has made a radio documentary about this study and the people who took part. Lawrynuik’s April 16, 2019 article describes some highlights from the radio documentary,

Imagine waking up in a hospital bed surrounded by the beeps and whirring sounds of the machines keeping you alive. The doctor tells you that you will likely never walk again.

But then, just as you begin to process that news, a physiotherapist shows up at your bedside and says, “Hold up. I might have a special opportunity for you.”

That’s the journey taken by a number of Albertans who landed in Calgary’s Foothills Medical Centre after accidents or trauma to their spine in the last three years. Three of those people are Alex McEwan, a university student in Lethbridge; Jean Ogilvie, a 77-year-old woman living in Calgary; and Josh Pelland, a former climber turned motivational speaker in Three Hills, Alta.

All three are united by a technology called an exoskeleton, created by a company called Ekso Bionics, that allowed them to walk despite no longer being able to use their legs. 

“The first time was a bit scary actually,” Ogilvie said. “It’s like a great big skeleton that sort of clasps you in its body. [It’s] black and all sorts of straps and sensors tell you how I’m doing.”

Pelland agrees about how daunting the experience is to start.

“They just said, ‘OK, the machine is going to assist you and lift you up.’ And I was a bit like, ‘OK, this is the strangest thing ever.'”

Once the frame of the exoskeleton is strapped along the outside of the patient’s legs and up their back, starting from the seated position, it does lift them completely without the help of their own muscles.

From there they shift their upper-body weight within the machine to hit certain targets — once your body weight is shifted forward and laterally enough, a beep sounds and the exoskeleton pulls each leg forward, one at a time. 

As patients learn to use the machine, they walk with the assistance of a walker. Then, as they progress, they upgrade to forearm crutches. The entire time, they’re accompanied by the man behind the machine, Kyle McIntosh.

McIntosh is a physiotherapist and he worked with the exoskeleton both to help patients and to conduct research into the machine’s impact on rehabilitation.

After being discharged and living once again without the exoskeleton, and therefore without the ability to walk — McEwan got an idea: maybe he’d be allowed to use the robot, just one last time.

“High school wasn’t high school for me. I only really got one semester of grade 10 before I broke my spine. So that first semester was great. I enjoyed it. I played sports. I was a good student. But then it was no longer about high school anymore. It was more about adjusting to my new life.”

McIntosh and McEwan hatched the plan together and kept it a closely guarded secret. Then, on the day McEwan was set to graduate from Grade 12, he asked to be placed last on the list of students to cross the stage.

“I remember taking a first few steps and not hearing very much. Hearing people cheer because I was the kid in the wheelchair at the high school, so it makes sense. But the second they saw the canes and my first few steps, just one kid erupted: ‘Yeah!’ And then everyone went crazy.”

“I think walking across the stage — just like I got to walk into my high school on the first day of Grade 10 — was a really good closing story. The chapter of me learning to live in a wheelchair was done. And it was now my turn to go live my life. So that’s why I think it was such an important day because it gave me a lot of closure. I got to walk into the high school, I got to walk out.”

If you have the time, you might want to read Lawrynuik’s April 16, 2019 article in its entirety. It turns out that the study did much more than give a people a chance to walk again, even if just for a short time.

Anyone interested in the robotic, wearable exoskeleton used in the study can go here to EksoHealth, the company that produces the EksoGT, a bionic exoskeleton. (Lawrynuik’s article has another name for the product, i.e., Ekso Bionic Exoskeleton but all I could find was the EksoGT.)

Brazil and Walk Again

The most recent post featuring the Walk Again project is my May 20, 2014 edition which was part of a larger series on ‘Brain research, ethics, and nanotechnology’. The May 20, 2014 posting covered Walk Again’s debut at the 2014 World Cup (soccer/football) in Brazil. Unfortunately,, the lead researcher Miguel Nicolelis oversold the technology. I think people were expecting someone with paraplegia to come bounding out onto the field and give a flashy opening kick for the tournament what they saw was something a great deal more restrained.

The person was wheeled out onto the field, stood up, shuffled a bit, and nudged the ball with his foot. It represented a huge breakthrough but it wasn’t flashy.

The latest from Walk Again is in a May 14, 2019 Associação Alberto Santos Dumont para Apoio à Pesquisa press release on EurekAlert,

In another major clinical breakthrough of the Walk Again Project, a non-profit international consortium aimed at developing new neuro-rehabilitation protocols, technologies and therapies for spinal cord injury, two patients with paraplegia regained the ability to walk with minimal assistance, through the employment of a fully non-invasive brain-machine interface that does not require the use of any invasive spinal cord surgical procedure. The results of this study appeared on the May 1 [2019] issue of the journal Scientific Reports.

The two patients with paraplegia (AIS C) used their own brain activity to control the non-invasive delivery of electrical pulses to a total of 16 muscles (eight in each leg), allowing them to produce a more physiological walk than previously reported, requiring only a conventional walker and a body weight support system as assistive devices. Overall, the two patients were able to produce more than 4,500 steps using this new technology, which combines a non-invasive brain-machine interface, based on a 16-channel EEG, to control a multi-channel functional electrical stimulation system (FES), tailored to produce a much smoother gait pattern than the state of the art of this technique.

“What surprised us was that, in addition to allowing these patients to walk with little help, one of them displayed a clear motor improvement by practicing with this new approach. Patients required approximatively [sic] 25 sessions to master the training before they were able to walk using this apparatus,” said Solaiman Shokur one of the authors of the study.

The two patients that used this new rehabilitation approach had previously participated in the long-term neurorehabilitation study carried out using the Walk Again Project Neurorehabilitation (WANR) protocol. As reported in a recent publication from the same team (Shokur et al., PLoS One, Nov. 2018), all seven patients who participated in that protocol for a period of 28 months improved their clinical status, from complete paraplegia (AIS A or B, meaning no motor functions below the level of the injury, according to the ASIA classification) to partial paraplegia (AIS C, meaning partial recovery of sensory and motor function below the injury level). This significant neurological recovery included major clinical improvements in sensory discrimination (tactile, nociception, vibration, and pressure), voluntary motor control of abdomen and leg muscles, and important gains in autonomic control, such as bladder, bowel, and sexual functions.

“The last two studies published by the Walk Again Project clearly indicate that partial neurological and functional recovery can be induced in chronic spinal cord injury patients by combining multiple non-invasive technologies that are based around the concept of using a brain-machine interface to control different types of actuators, like virtual avatars, robotic walkers, or muscle stimulating devices, to allow the total involvement of patients in their own rehabilitation routine,” said Miguel Nicolelis, scientific director of the Walk Again Project and one of the authors of the study.

In a recent report by another group, one AIS C and two AIS D patients were able to walk thanks to the employment of an invasive method for spinal cord electrical stimulation, which required a spinal surgical procedure. In contrast, in the present study two AIS C patients – which originally were AIS A (see Supplemental Material below)- and a third AIS B subject, who recently achieved similar results, were able to regain a significant degree of autonomous walking without the need for such invasive treatments. Instead, these patients only received electrical stimulation patterns delivered to the skin surface of their legs, so that a total of eight muscles in each limb could be electrically stimulated in a physiologically accurate sequence. This was done in order to produce a smoother and more natural pattern of locomotion.

“Crucial for this implementation was the development of a closed-loop controller that allowed real-time correction of the patients’ walking pattern, taking into account muscle fatigue and external perturbations, in order to produce a predefined gait trajectory. Another major component of our approach was the use of a wearable haptic display to deliver tactile feedback to the patients´ forearms in order to provide them with a continuous source of proprioceptive feedback related to their walking,” said Solaiman Shokur.

To control the pattern of electrical muscle stimulation in each leg, these patients utilized an EEG-based brain-machine interface. In this setup, patients learned to alternate the generation of “stepping motor imagery” activity in their right and left motor cortices, in order to create alternated movements of their left and right legs.

According to the authors, the patients exhibited not only “less dependency on walking assistance, but also partial neurological recovery, with substantial rates of motor improvement in one of them.” The improvement in motor control in this last AIS C patient was 9 points in the lower extremity motor score (LEMS), which was comparable with that observed using invasive spinal cord stimulation.

Based on the results obtained over the past 5 years, the WAP now intends to combine all its neurorehabilitation tools into a single integrated, non-invasive platform to treat spinal cord injury patients. This platform will allow patients to begin training soon after the injury occurs. It will also allow the employment of a multi-dimensional integrated brain-machine interface capable of simultaneously controlling virtual and robotic actuators (like a lowerlimb exoskeleton), a multi-channel non-invasive electrical muscle stimulation system (like the FES used in the present study), and a novel non-invasive spinal cord stimulation approach. In this final configuration, this WAP platform will incorporate all these technologies together in order to maximize neurological and functional recovery in the shortest possible time, without the need of any invasive procedure.

According to Dr. Nicolelis, “there is no silver bullet to treat spinal cord injuries. More and more, it looks like we need to implement multiple techniques simultaneously to achieve the best neurorehabilitation results. In this context, it is also imperative to consider the occurrence of cortical plasticity as a major component in the planning of our rehabilitation approach.”

Here’s a link to and a citation for the paper,

Non-invasive, Brain-controlled Functional Electrical Stimulation for Locomotion Rehabilitation in Individuals with Paraplegia by Aurelie Selfslagh, Solaiman Shokur, Debora S. F. Campos, Ana R. C. Donati, Sabrina Almeida, Seidi Y. Yamauti, Daniel B. Coelho, Mohamed Bouri & Miguel A. L. Nicolelis. Scientific Reports volume 9, Article number: 6782 (2019) DOI: https://doi.org/10.1038/s41598-019-43041-9 Published 01 May 2019

This paper is open access.

There’s also a video for Walk Again,

TED Global would like to see you in Rio—USD $6,000 + application required

TED (technology, entertainment, design) Global is being held in Rio de Janeiro, Brazil in October 2014 and there are still a few spots left for participants according to a July 23, 2014 notice (I checked here, there are still openings as of Aug. 1, 2014),

In early October, Rio de Janeiro will host our first TEDGlobal in South America. The conference theme is “South” and you can meet here. Held in the historic Copacabana Palace Hotel on the eponymous beach, TEDGlobal 2014 promises speakers with amazing new ideas to stimulate your mind, while the rest of you takes in the beauty that is Rio: the ocean, the beach, the volcanic mountains, and the energetic Cariocas. It is simply one of the most beautiful cities on Earth.

We hope you will join us at this more intimately scaled event (half the size of TED in Vancouver), and celebrate ideas from across the Global South.

The conference takes place October 5-10, 2014. During five immersive days of talks, music, performances, tech demos, exhibits and wonderful parties, the conference will focus on the Global South’s rise in influence and power — plus relevant stories from the rest of the world.

A small number of passes remain for $6,000 and $12,000. …

Questions? Email registration@ted.com.

Vê-lo no Rio (See you in Rio)

There is a list of their currently confirmed speakers here. It includes:

Grimanesa Amoros, Peruvian interdisciplinary artist

Séverine Autesserre, Congo scholar
Tasso Azevedo, Brazilian forest conservationist
Rodrigo Baggio, Brazilian digital inclusionist
Khalida Brohi, Pakistani equality activist

Wendy Freedman, Astronomer

Syed Karim, Satellite datacaster
...
Miguel Nicolelis, Brain interface pioneer

Mark Plotkin, Amazonian ethnobotanist

Matthieu Ricard, Buddhist monk

Steve Song, Africa connectivity tinkerer
Jorge Soto, Cancer detection technologist

Zeynep Tufekci, Technosociologist

Tashka Yawanawa, Amazonian chief

I recognized two names on the full list: Miguel Nicolelis (featured here many times and most recently in a May 20, 2014 posting) and Matthieu Ricard (mentioned here once.in an April 11, 2013 posting). Both of them were mentioned in regard to the field of neuroscience.

On that note, Happy Weekend on what is a long weekend for many Canadians including me!

Brains, prostheses, nanotechnology, and human enhancement: summary (part five of five)

The Brain research, ethics, and nanotechnology (part one of five) May 19, 2014 post kicked off a series titled ‘Brains, prostheses, nanotechnology, and human enhancement’ which brings together a number of developments in the worlds of neuroscience, prosthetics, and, incidentally, nanotechnology in the field of interest called human enhancement. Parts one through four are an attempt to draw together a number of new developments, mostly in the US and in Europe. Due to my language skills which extend to English and, more tenuously, French, I can’t provide a more ‘global perspective’.

Now for the summary. Ranging from research meant to divulge more about how the brain operates in hopes of healing conditions such as Parkinson’s and Alzeheimer’s diseases to utilizing public engagement exercises (first developed for nanotechnology) for public education and acceptance of brain research to the development of prostheses for the nervous system such as the Walk Again robotic suit for individuals with paraplegia (and, I expect quadriplegia [aka tetraplegia] in the future), brain research is huge in terms of its impact socially and economically across the globe.

Until now, I have not included information about neuromorphic engineering (creating computers with the processing capabilities of human brains). My May 16, 2014 posting (Wacky oxide. biological synchronicity, and human brainlike computing) features one of the latest developments along with this paragraph providing links to overview materials of the field,

As noted earlier, there are other approaches to creating an artificial brain, i.e., neuromorphic engineering. My April 7, 2014 posting is the most recent synopsis posted here; it includes excerpts from a Nanowerk Spotlight article overview along with a mention of the ‘brain jelly’ approach and a discussion of my somewhat extensive coverage of memristors and a mention of work on nanoionic devices. There is also a published roadmap to neuromorphic engineering featuring both analog and digital devices, mentioned in my April 18, 2014 posting.

There is an international brain (artificial and organic) enterprise underway. Meanwhile, work understanding the brain will lead to new therapies and, inevitably, attempts to enhance intelligence. There are already drugs and magic potions (e.g. oxygenated water in Mental clarity, stamina, endurance — is it in the bottle? Celebrity athletes tout the benefits of oxygenated water, but scientists have their doubts, a May 16,2014 article by Pamela Fayerman for the Vancouver Sun). In a June 19, 2009 posting featured Jamais Cascio’s  speculations about augmenting intelligence in an Atlantic magazine article.

While researchers such Miguel Nicolelis work on exoskeletons (externally worn robotic suits) controlled by the wearer’s thoughts and giving individuals with paraplegia the ability to walk, researchers from one of Germany’s Fraunhofer Institutes reveal a different technology for achieving the same ends. From a May 16, 2014 news item on Nanowerk,

People with severe injuries to their spinal cord currently have no prospect of recovery and remain confined to their wheelchairs. Now, all that could change with a new treatment that stimulates the spinal cord using electric impulses. The hope is that the technique will help paraplegic patients learn to walk again. From June 3 – 5 [2-14], Fraunhofer researchers will be at the Sensor + Test measurement fair in Nürnberg to showcase the implantable microelectrode sensors they have developed in the course of pre-clinical development work (Hall 12, Booth 12-537).

A May 14, 2014 Fraunhofer Institute news release, which originated the news item, provides more details about this technology along with an image of the implantable microelectrode sensors,

The implantable microelectrode sensors are flexible and wafer-thin. © Fraunhofer IMM

The implantable microelectrode sensors are flexible and wafer-thin.
© Fraunhofer IMM

Now a consortium of European research institutions and companies want to get affected patients quite literally back on their feet. In the EU’s [European Union’s] NEUWalk project, which has been awarded funding of some nine million euros, researchers are working on a new method of treatment designed to restore motor function in patients who have suffered severe injuries to their spinal cord. The technique relies on electrically stimulating the nerve pathways in the spinal cord. “In the injured area, the nerve cells have been damaged to such an extent that they no longer receive usable information from the brain, so the stimulation needs to be delivered beneath that,” explains Dr. Peter Detemple, head of department at the Fraunhofer Institute for Chemical Technology’s Mainz branch (IMM) and NEUWalk project coordinator. To do this, Detemple and his team are developing flexible, wafer-thin microelectrodes that are implanted within the spinal canal on the spinal cord. These multichannel electrode arrays stimulate the nerve pathways with electric impulses that are generated by the accompanying by microprocessor-controlled neurostimulator. “The various electrodes of the array are located around the nerve roots responsible for locomotion. By delivering a series of pulses, we can trigger those nerve roots in the correct order to provoke motion sequences of movements and support the motor function,” says Detemple.

Researchers from the consortium have already successfully conducted tests on rats in which the spinal cord had not been completely severed. As well as stimulating the spinal cord, the rats were given a combination of medicine and rehabilitation training. Afterwards the animals were able not only to walk but also to run, climb stairs and surmount obstacles. “We were able to trigger specific movements by delivering certain sequences of pulses to the various electrodes implanted on the spinal cord,” says Detemple. The research scientist and his team believe that the same approach could help people to walk again, too. “We hope that we will be able to transfer the results of our animal testing to people. Of course, people who have suffered injuries to their spinal cord will still be limited when it comes to sport or walking long distances. The first priority is to give them a certain level of independence so that they can move around their apartment and look after themselves, for instance, or walk for short distances without requiring assistance,” says Detemple.

Researchers from the NEUWalk project intend to try out their system on two patients this summer. In this case, the patients are not completely paraplegic, which means there is still some limited communication between the brain and the legs. The scientists are currently working on tailored implants for the intervention. “However, even if both trials are a success, it will still be a few years before the system is ready for the general market. First, the method has to undergo clinical studies and demonstrate its effectiveness among a wider group of patients,” says Detemple.

Patients with Parkinson’s disease could also benefit from the neural prostheses. The most well-known symptoms of the disease are trembling, extreme muscle tremors and a short, [emphasis mine] stooped gait that has a profound effect on patients’ mobility. Until now this neurodegenerative disorder has mostly been treated with dopamine agonists – drugs that chemically imitate the effects of dopamine but that often lead to severe side effects when taken over a longer period of time. Once the disease has reached an advanced stage, doctors often turn to deep brain stimulation. This involves a complex operation to implant electrodes in specific parts of the brain so that the nerve cells in the region can be stimulated or suppressed as required. In the NEUWalk project, researchers are working on electric spinal cord simulation – an altogether less dangerous intervention that should however ease the symptoms of Parkinson’s disease just as effectively. “Initial animal testing has yielded some very promising results,” says Detemple.

(For anyone interested in the NEUWalk project, you can find more here,) Note the reference to Parkinson’s in the context of work designed for people with paraplegia. Brain research and prosthetics (specifically neuroprosthetics or neural prosthetics), are interconnected. As for the nanotechnology connection, in its role as an enabling technology it has provided some of the tools that make these efforts possible. It has also made some of the work in neuromorphic engineering (attempts to create an artificial brain that mimics the human brain) possible. It is a given that research on the human brain will inform efforts in neuromorphic engineering and that attempts will be made to create prostheses for the brain (cyborg brain) and other enhancements.

One final comment, I’m not so sure that transferring approaches and techniques developed to gain public acceptance of nanotechnology are necessarily going to be effective. (Harthorn seemed to be suggesting in her presentation to the Presidential Presidential Commission for the Study of Bioethical Issues that these ‘nano’ approaches could be adopted. Other researchers [Caulfield with the genome and Racine with previous neuroscience efforts] also suggested their experience could be transferred. While some of that is likely true,, it should be noted that some self-interest may be involved as brain research is likely to be a fresh source of funding for social science researchers with experience in nanotechnology and genomics who may be finding their usual funding sources less generous than previously.)

The likelihood there will be a substantive public panic over brain research is higher than it ever was for a nanotechnology panic (I am speaking with the benefit of hindsight re: nano panics). Everyone understands the word, ‘brain’, far fewer understand the word ‘nanotechnology’ which means that the level of interest is lower and people are less likely to get disturbed by an obscure technology. (The GMO panic gained serious traction with the ‘Frankenfood’ branding and when it fused rather unexpectedly with another research story,  stem cell research. In the UK, one can also add the panic over ‘mad cow’ disease or Creutzfeldt-Jakob disease (CJD), as it’s also known, to the mix. It was the GMO and other assorted panics which provided the impetus for much of the public engagement funding for nanotechnology.)

All one has to do in this instance is start discussions about changing someone’s brain and cyborgs and these researchers may find they have a much more volatile situation on their hands. As well, everyone (the general public and civil society groups/activists, not just the social science and science researchers) involved in the nanotechnology public engagement exercises has learned from the experience. In the meantime, pop culture concerns itself with zombies and we all know what they like to eat.

Links to other posts in the Brains, prostheses, nanotechnology, and human enhancement five-part series

Part one: Brain research, ethics, and nanotechnology (May 19, 2014 post)

Part two: BRAIN and ethics in the US with some Canucks (not the hockey team) participating (May 19, 2014)

Part three: Gray Matters: Integrative Approaches for Neuroscience, Ethics, and Society issued May 2014 by US Presidential Bioethics Commission (May 20, 2014)

Part four: Brazil, the 2014 World Cup kickoff, and a mind-controlled exoskeleton (May 20, 2014)

Brazil, the 2014 World Cup kickoff, and a mind-controlled exoskeleton (part four of five)

The Brain research, ethics, and nanotechnology (part one of five) May 19, 2014 post kicked off a series titled ‘Brains, prostheses, nanotechnology, and human enhancement’ which brings together a number of developments in the worlds of neuroscience, prosthetics, and, incidentally, nanotechnology in the field of interest called human enhancement. Parts one through four are an attempt to draw together a number of new developments, mostly in the US and in Europe. Due to my language skills which extend to English and, more tenuously, French, I can’t provide a more ‘global perspective’. Part five features a summary.

Brazil’s World Cup for soccer/football which opens on June 12, 2014 will be the first public viewing of someone with paraplegia demonstrating a mind-controlled exoskeleton (or a robotic suit as it’s sometimes called) by opening the 2014 games with the first kick-off.

I’ve been covering this story since 2011 and, even so, was late to the party as per this May 7, 2014 article by Alejandra Martins for BBC World news online,

The World Cup curtain-raiser will see the first public demonstration of a mind-controlled exoskeleton that will enable a person with paralysis to walk.

If all goes as planned, the robotic suit will spring to life in front of almost 70,000 spectators and a global audience of billions of people.

The exoskeleton was developed by an international team of scientists as part of the Walk Again Project and is the culmination of more than a decade of work for Dr Miguel Nicolelis, a Brazilian neuroscientist based at Duke University in North Carolina. [emphasis mine]

Since November [2013], Dr Nicolelis has been training eight patients at a lab in Sao Paulo, in the midst of huge media speculation that one of them will stand up from his or her wheelchair and deliver the first kick of this year’s World Cup.

“That was the original plan,” the Duke University researcher told the BBC. “But not even I could tell you the specifics of how the demonstration will take place. This is being discussed at the moment.”

Speaking in Portuguese from Sao Paulo, Miguel Nicolelis explained that all the patients are over 20 years of age, with the oldest about 35.

“We started the training in a virtual environment with a simulator. In the last few days, four patients have donned the exoskeleton to take their first steps and one of them has used mental control to kick a ball,” he explained.

The history of Nicolelis’ work is covered here in a series of a posts starting the with an Oct. 5, 2011 post (Advertising for the 21st Century: B-Reel, ‘storytelling’, and mind control; scroll down 2/3 of the way for a reference to Ed Yong’s article where I first learned of Nicolelis).

The work was explored in more depth in a March 16, 2012 posting (Monkeys, mind control, robots, prosthetics, and the 2014 World Cup (soccer/football) and then followed up a year later by two posts which link Nicoleliis’ work with the Brain Activity Map (now called, BRAIN [Brain Research through Advancing Innovative Neurotechnologies] initiative: a March 4, 2013 (Brain-to-brain communication, organic computers, and BAM [brain activity map], the connectome) and a March 8,  2013 post (Prosthetics and the human brain) directly linking exoskeleton work in Holland and the project at Duke with current brain research and the dawning of a new relationship to one’s prosthestics,

On the heels of research which suggests that humans tend to view their prostheses, including wheel chairs, as part of their bodies, researchers in Europe  have announced the development of a working exoskeleton powered by the wearer’s thoughts.

Getting back to Brazil and Nicolelis’ technology, Ian Sample offers an excellent description in an April 1, 2014 article for the Guardian (Note: Links have been removed),

The technology in question is a mind-controlled robotic exoskeleton. The complex and conspicuous robotic suit, built from lightweight alloys and powered by hydraulics, has a simple enough function. When a paraplegic person straps themselves in, the machine does the job that their leg muscles no longer can.

The exoskeleton is the culmination of years of work by an international team of scientists and engineers on the Walk Again project. The robotics work was coordinated by Gordon Cheng at the Technical University in Munich, and French researchers built the exoskeleton. Nicolelis’s team focused on ways to read people’s brain waves, and use those signals to control robotic limbs.

To operate the exoskeleton, the person is helped into the suit and given a cap to wear that is fitted with electrodes to pick up their brain waves. These signals are passed to a computer worn in a backpack, where they are decoded and used to move hydraulic drivers on the suit.

The exoskeleton is powered by a battery – also carried in the backpack – that allows for two hours of continuous use.

“The movements are very smooth,” Nicolelis told the Guardian. “They are human movements, not robotic movements.”

Nicolelis says that in trials so far, his patients seem have taken to the exoskeleton. “This thing was made for me,” one patient told him after being strapped into the suit.

The operator’s feet rest on plates which have sensors to detect when contact is made with the ground. With each footfall, a signal shoots up to a vibrating device sewn into the forearm of the wearer’s shirt. The device seems to fool the brain into thinking that the sensation came from their foot. In virtual reality simulations, patients felt that their legs were moving and touching something.

Sample’s article includes a good schematic of the ‘suit’ which I have not been able to find elsewhere (meaning the Guardian likely has a copyright for the schematic and is why you won’t see it here) and speculation about robotics and prosthetics in the future.

Nicolelis and his team have a Facebook page for the Walk Again Project where you can get some of the latest information with  both English and Portuguese language entries as they prepare for the June 12, 2014 kickoff.

One final thought, this kickoff project represents an unlikely confluence of events. After all, what are the odds

    • that a Brazil-born researcher (Nicolelis) would be working on a project to give paraplegics the ability to walk again? and
    • that Brazil would host the World Cup in 2014 (the first time since 1950)? and
    • that the timing would coincide so a public demonstration at one of the world’s largest athletic events (of a sport particularly loved in Brazil) could be planned?

It becomes even more extraordinary when one considers that Brazil had isolated itself somewhat in the 1980s with a policy of nationalism vis à vis the computer industry (from the Brazil Science and Technology webpage on the ITA website),

In the early 1980s, the policy of technological nationalism and self-sufficiency had narrowed to the computer sector, where protective legislation tried to shield the Brazilian mini- and microcomputer industries from foreign competition. Here again, the policy allowed for the growth of local industry and a few well-qualified firms, but the effect on the productive capabilities of the economy as a whole was negative; and the inability to follow the international market in price and quality forced the policy to be discontinued.

For those who may have forgotten, the growth of the computer industry (specifically personal computers) in the 1980s figured hugely in a country’s economic health and, in this case,with  a big negative impact in Brazil.

Returning to 2014, the kickoff in Brazil (if successful) symbolizes more than an international athletic competition or a technical/medical achievement, this kick-off symbolizes a technological future for Brazil and its place on the world stage (despite the protests and social unrest) .

Links to other posts in the Brains, prostheses, nanotechnology, and human enhancement five-part series

Part one: Brain research, ethics, and nanotechnology (May 19, 2014 post)

Part two: BRAIN and ethics in the US with some Canucks (not the hockey team) participating (May 19, 2014)

Part three: Gray Matters: Integrative Approaches for Neuroscience, Ethics, and Society issued May 2014 by US Presidential Bioethics Commission (May 20, 2014)

Part five: Brains, prostheses, nanotechnology, and human enhancement: summary (May 20, 2014)

ETA June 16, 2014: The kickoff seems to have been a disappointment (June 15, 2014 news item on phys.org) and for those who might be interested in some of the reasons for the World Cup unrest and protests in Brazil, John Oliver provides an excoriating overview of the organization which organizes the World Cup games while professing his great love of the games, http://www.youtube.com/watch?v=DlJEt2KU33I

Listening to an individual brain cell using a carbon nanotube ‘harpoon’

Apparently, the prime motivation for listening to individual neurons or brain cells is to “better understand the computational complexity of the brain,” according to a June 20,  2013 news item on Azonano,

The new brain cell spear is a millimeter long, only a few nanometers wide and harnesses the superior electromechanical properties of carbon nanotubes to capture electrical signals from individual neurons.

“To our knowledge, this is the first time scientists have used carbon nanotubes to record signals from individual neurons, what we call intracellular recordings, in brain slices or intact brains of vertebrates,” said Bruce Donald, a professor of computer science and biochemistry at Duke University who helped developed the probe.

The June 19, 2013 Duke University news release by Ashley Yeager, which originated the news item, provides some insight into the current state of the art and how this new technique is an improvement,

Currently, they use two main types of electrodes, metal and glass, to record signals from brain cells. Metal electrodes record spikes from a population of brain cells and work well in live animals. Glass electrodes also measure spikes, as well as the computations individual cells perform, but are delicate and break easily.”The new carbon nanotubes combine the best features of both metal and glass electrodes. They record well both inside and outside brain cells, and they are quite flexible. Because they won’t shatter, scientists could use them to record signals from individual brain cells of live animals,” said Duke neurobiologist Michael Platt, who was not involved in the study.

This is not the first time researchers have tried to use carbon nanotubes for this purpose, from the news release,

In the past, other scientists have experimented with carbon nanotube probes. But the electrodes were thick, causing tissue damage, or they were short, limiting how far they could penetrate into brain tissue. They could not probe inside individual neurons.

To change this, Donald began working on a harpoon-like carbon-nanotube probe with Duke neurobiologist Richard Mooney five years ago. The two met during their first year at Yale in the 1976, kept in touch throughout graduate school and began meeting to talk about their research after they both came to Duke.

Mooney told Donald about his work recording brain signals from live zebra finches and mice. The work was challenging, he said, because the probes and machinery to do the studies were large and bulky on the small head of a mouse or bird.

With Donald’s expertise in nanotechnology and robotics and Mooney’s in neurobiology, the two thought they could work together to shrink the machinery and improve the probes with nano-materials.

To make the probe, graduate student Inho Yoon and Duke physicist Gleb Finkelstein used the tip of an electrochemically sharpened tungsten wire as the base and extended it with self-entangled multi-wall carbon nanotubes to create a millimeter-long rod. The scientists then sharpened the nanotubes into a tiny harpoon using a focused ion beam at North Carolina State University.

Yoon then took the nano-harpoon to Mooney’s lab and jabbed it into slices of mouse brain tissue and then into the brains of anesthetized mice. The results show that the probe transmits brain signals as well as, and sometimes better than, conventional glass electrodes and is less likely to break off in the tissue. The new probe also penetrates individual neurons, recording the signals of a single cell rather than the nearest population of them.

Based on the results, the team has applied for a patent on the nano-harpoon.  Platt said scientists might use the probes in a range of applications, from basic science to human brain-computer interfaces and brain prostheses.

Donald said the new probe makes advances in those directions, but the insulation layers, electrical recording abilities and geometry of the device still need improvement.

The research paper is available in the open access journal PLoS ONE,

Intracellular Neural Recording with Pure Carbon Nanotube Probes by Inho Yoon, Kosuke Hamaguchi, Ivan V. Borzenets, Gleb Finkelstein, Richard Mooney, and Bruce R. Donald. 2013. PLOS ONE. DOI: 10.1371/journal.pone.0065715

As for calling this a ‘harpoon’, these carbon nanotube probes really do resemble harpoons,

This image, taken with a scanning electron microscope, shows a new brain electrode that tapers to a point as thick as a single carbon nanotube. Credit: Inho Yoon and Bruce Donald, Duke.  [downloaded from http://today.duke.edu/2013/06/brainharpoon]

This image, taken with a scanning electron microscope, shows a new brain electrode that tapers to a point as thick as a single carbon nanotube. Credit: Inho Yoon and Bruce Donald, Duke. [downloaded from http://today.duke.edu/2013/06/brainharpoon]

You can compare it to this harpoon from The Specialists Prop House, Traditional harpoon page,

[downloaded from The Specialists Prop House, Traditional harpoon page, http://thespecialistsltd.com/traditional-harpoon]

[downloaded from The Specialists Prop House, Traditional harpoon page, http://thespecialistsltd.com/traditional-harpoon]

I have written about some of the neuroscience work coming out of Duke University in the past, e.g., my March 4, 2013 posting about Miguel Nicolelis’ work on brain-to-brain communication.

Brazil, Canada, and an innovation, science, and technology forum in Vancouver (Canada)

The Brazil-Canada Chamber of Commerce (BCCC) is presenting, in partnership with Simon Fraser University’s (SFU) Beedie School of Business, an all-morning forum on June 17, 2013. From the SFU Vancouver Events: June 14 – 21, 2013 announcement (Note: Links have been removed),

Monday, June 17 [2013]

Brazil-Canada Business, Innovation, Science, and Technology Forum

Time: 8-11:30am

Place: Segal Graduate Business School, 500 Granville St.

Cost: $35-70, register online

Join us for a morning focused on Business Innovation and Science & Tecnology opportunities in the Brazilian economy. The opening speakers, Ambassador Sergio Florencio, Consul General and Dr. Jeremy Hall will provide an overview of the landscape in Brazil. The panel discussion includes industry leaders who have piloted extensive business in Brazil specifically in the agriculture, mining and infrastructure fields: Marcelo Sarkis, Heenan Blaikie; Ray Castelli, Weatherhaven and Rogerio Tippe, Javelin Partners. If you are interested in conducting business in Brazil and would like to understand more about the dynamics of the Brazilian economy and how businesses operate, please register now.

If the event is about business, innovation, science, and technology, it seems curious the only mentions of science and/or technology in the event description are confined to a few of the panelists’ interests in agriculture, mining, and whatever they mean by infrastructure.

Brazil is one of the BRICS (Brazil, Russia,India, China, and South Africa) countries and, from what I understand, this very loose coalition is eager to take a leadership position vis à vis science, technology, and innovation supplanting the dominance of the US, Japan, and the European Union.

In the early 1990s, I wrote a paper about science and technology transfer and noted that Brazil was entering a new period of development after years of the country’s science and technology efforts (scientists) being isolated from the rest of the world in a failed  attempt to create a powerhouse international enterprise.

Some 20 years later, the decision to join the rest of the science and technology world seems to have been successful. Brazil is set to host the 2014 World Cup for soccer (or, as most of the world calls it, football) and the summer Olympics in 2016. (Sports are often correlated with science and technology advances.) I don’t believe any other country has ever attempted to host two such large international sports events within two years of each other. That’s a pretty confident attitude.

There are two areas of science and technology research in Brazil that are of particular interest to me, brain research and the work on cellulose nanocrystals (CNC), also known as, nanocrystalline cellulose (NCC).

While the focus was on Miguel Nicolelis and Duke University (US), the recent announcement of brain-to-brain communication via the Internet featured a research facility in Brazil (from my Mar. 4, 2013 posting),

Miguel Nicolelis, a professor at Duke University, has been making international headlines lately with two brain projects. The first one about implanting a brain chip that allows rats to perceive infrared light was mentioned in my Feb. 15, 2013 posting. The latest project is a brain-to-brain (rats) communication project as per a Feb. 28, 2013 news release on *EurekAlert,

Researchers have electronically linked the brains of pairs of rats for the first time, enabling them to communicate directly to solve simple behavioral puzzles. A further test of this work successfully linked the brains of two animals thousands of miles apart—one in Durham, N.C., and one in Natal, Brazil.

The results of these projects suggest the future potential for linking multiple brains to form what the research team is calling an “organic computer,” which could allow sharing of motor and sensory information among groups of animals. The study was published Feb. 28, 2013, in the journal Scientific Reports.

“Our previous studies with brain-machine interfaces had convinced us that the rat brain was much more plastic than we had previously thought,” said Miguel Nicolelis, M.D., PhD, lead author of the publication and professor of neurobiology at Duke University School of Medicine. “In those experiments, the rat brain was able to adapt easily to accept input from devices outside the body and even learn how to process invisible infrared light generated by an artificial sensor. So, the question we asked was, ‘if the brain could assimilate signals from artificial sensors, could it also assimilate information input from sensors from a different body?’”

One of Nicolelis’s other goals is to have someone with quadriplegia kick the opening ball for the Brazil-hosted 2014 World Cup (Walk Again Project). From my Mar. 16, 2012 posting,

It is the exoskeleton described on the Walk Again Project home page that Nicolelis is hoping will enable a young Brazilian quadriplegic to deliver the opening kick for the 2014 World Cup (soccer/football) in Brazil.

Moving on to the other area of interest, CNC research , which in Canada is discussed in terms of the forestry industry (I’ve blogged about this extensively, the search term NCC should fetch most if not all of my postings on the topic), is taking a different tack in Brazil where the focus is on pineapple and banana fibres. My Mar. 28, 20111 posting (Nanocellulose fibres, pineapples, bananas, and cars) focuses on cellulose and plastic,

Brazilian researchers are working on ways to use nanocellulose fibres from various plants to reinforce plastics in the automotive industry. From the March 28, 2011 news item on Nanowerk,

Study leader Alcides Leão, Ph.D., said the fibers used to reinforce the new plastics may come from delicate fruits like bananas and pineapples, but they are super strong. Some of these so-called nano-cellulose fibers are almost as stiff as Kevlar, the renowned super-strong material used in armor and bulletproof vests. Unlike Kevlar and other traditional plastics, which are made from petroleum or natural gas, nano-cellulose fibers are completely renewable.

My second and, to date, only other posting (June 16, 2011) about the work in Brazil features a transcript of an interview with CNC researcher, Alcides Leão.

Finally, I have a few factoids which I will tie together, loosely, and try to show how they relate to this forum. First, São Paulo, Brazil hosts the world’s second oldest and one of its most important biennial visual arts events. (BTW, the next one, Bienal de São Paulo,  is in 2014.) Second, the recent Council of Canadian Academies assessment, State of Science and Technology in Canada, 2012, stated that Canada rates very highly in six areas, one of those areas being the Visual and Performing Arts. Admittedly Canada’s prominence in the visual and performing is fueled largely by efforts in Québec (as per the assessment), still, one would think there might be some value in trying to include that sector in this  forum and encourage the local visual and performing arts technology industry to make connections with the Brazilian industry.

Finally for those of you who have persisted, here’s the link to buy tickets for the June 17, 2012 forum.

ETA June 21, 2013: The protests in Brazil have attracted worldwide attention and according to a June 21,2013 posting by Dillon Rand on Salon.com there are: 5 signs Brazil’s’ not ready to host the World Cup.

Prosthetics and the human brain

On the heels of research which suggests that humans tend to view their prostheses, including wheel chairs, as part of their bodies, researchers in Europe  have announced the development of a working exoskeleton powered by the wearer’s thoughts.

First, there’s the ‘wheelchair’ research, from the Mar. 6, 2013 news item on ScienceDaily,

People with spinal cord injuries show strong association of wheelchairs as part of their body, not extension of immobile limbs.

The human brain can learn to treat relevant prosthetics as a substitute for a non-working body part, according to research published March 6 in the open access journal PLOS ONE by Mariella Pazzaglia and colleagues from Sapienza University and IRCCS Fondazione Santa Lucia of Rome in Italy, supported by the International Foundation for Research in Paraplegie.

The researchers found that wheelchair-bound study participants with spinal cord injuries perceived their body’s edges as being plastic and flexible to include the wheelchair, independent of time since their injury or experience with using a wheelchair. Patients with lower spinal cord injuries who retained upper body movement showed a stronger association of the wheelchair with their body than those who had spinal cord impairments in the entire body.

According to the authors, this suggests that rather than being thought of only as an extension of the immobile limbs, the wheelchairs had become tangible, functional substitutes for the affected body part. …

As I mentioned in a Jan. 30, 2013 posting,

There have been some recent legal challenges as to what constitutes one’s body (from The Economist article, You, robot? [you can find the article here: http://www.economist.com/node/21560986]),

If you are dependent on a robotic wheelchair for mobility, for example, does the wheelchair count as part of your body? Linda MacDonald Glenn, an American lawyer and bioethicist, thinks it does. Ms Glenn (who is not involved in the RoboLaw project) persuaded an initially sceptical insurance firm that a “mobility assistance device” damaged by airline staff was more than her client’s personal property, it was an extension of his physical body. The airline settled out of court.

According to the Mar. 6, 2013 news release on EurekAlert from the Public Library of Science (PLoS), the open access article by Pazzaglia and her colleagues can be found here (Note: I have added a link),

Pazzaglia M, Galli G, Scivoletto G, Molinari M (2013) A Functionally Relevant Tool for the Body following Spinal Cord Injury. PLOS ONE 8(3): e58312.doi:10.1371/journal.pone.0058312

At almost the same time as Pazzaglia’s work,  a “Mind-controlled Exoskeleton” is announced in a Mar. 7, 2013 news item on ScienceDaily,

Every year thousands of people in Europe are paralysed by a spinal cord injury. Many are young adults, facing the rest of their lives confined to a wheelchair. Although no medical cure currently exists, in the future they could be able to walk again thanks to a mind-controlled robotic exoskeleton being developed by EU-funded researchers.

The system, based on innovative ‘Brain-neural-computer interface’ (BNCI) technology — combined with a light-weight exoskeleton attached to users’ legs and a virtual reality environment for training — could also find applications in there habilitation of stroke victims and in assisting astronauts rebuild muscle mass after prolonged periods in space.

The Mar. 7, 2013 news release on CORDIS, which originated the news item, offers a description of the “Mindwalker” project,

‘Mindwalker was proposed as a very ambitious project intended to investigate promising approaches to exploit brain signals for the purpose of controlling advanced orthosis, and to design and implement a prototype system demonstrating the potential of related technologies,’ explains Michel Ilzkovitz, the project coordinator at Space Applications Services in Belgium.

The team’s approach relies on an advanced BNCI system that converts electroencephalography (EEG) signals from the brain, or electromyography (EMG) signals from shoulder muscles, into electronic commands to control the exoskeleton.

The Laboratory of Neurophysiology and Movement Biomechanics at the Université Libre de Bruxelles (ULB) focused on the exploitation of EEG and EMG signals treated by an artificial neural network, while the Foundation Santa Lucia in Italy developed techniques based on EMG signals modelled by the coupling of neural and biomechanical oscillators.

One approach for controlling the exoskeleton uses so-called ‘steady-state visually evoked potential’, a method that reads flickering visual stimuli produced at different frequencies to induce correlated EEG signals. Detection of these EEG signals is used to trigger commands such as ‘stand’, ‘walk’, ‘faster’ or ‘slower’.

A second approach is based on processing EMG signals generated by the user’s shoulders and exploits the natural arm-leg coordination in human walking: arm-swing patterns can be perceived in this way and converted into control signals commanding the exoskeleton’s legs.

A third approach, ‘ideation’, is also based on EEG-signal processing. It uses the identification and exploitation of EEG Theta cortical signals produced by the natural mental process associated with walking. The approach was investigated by the Mindwalker team but had to be dropped due to the difficulty, and time needed, in turning the results of early experiments into a fully exploitable system.

Regardless of which method is used, the BNCI signals have to be filtered and processed before they can be used to control the exoskeleton. To achieve this, the Mindwalker researchers fed the signals into a ‘Dynamic recurrent neural network'(DRNN), a processing technique capable of learning and exploiting the dynamic character of the BNCI signals.

‘This is appealing for kinematic control and allows a much more natural and fluid way of controlling an exoskeleton,’ Mr Ilzkovitz says.

The team adopted a similarly practical approach for collecting EEG signals from the user’s scalp. Most BNCI systems are either invasive, requiring electrodes to be placed directly into brain tissue, or require users to wear a ‘wet’ capon their head, necessitating lengthy fitting procedures and the use of special gels to reduce the electrical resistance at the interface between the skin and the electrodes. While such systems deliver signals of very good quality and signal-to-noise ratio, they are impractical for everyday use.

The Mindwalker team therefore turned to a ‘dry’ technology developed by Berlin-based eemagine Medical Imaging Solutions: a cap covered in electrodes that the user can fit themselves, and which uses innovative electronic components to amplify and optimise signals before sending them to the neural network.

‘The dry EEG cap can be placed by the subject on their head by themselves in less than a minute, just like a swimming cap,’ Mr Ilzkovitz says.

Before proceeding any further with details, here’s what the Mindwalker looks like,

© MINDWALKER (downladed from http://cordis.europa.eu/fetch?CALLER=OFFR_TM_EN&ACTION=D&RCN=10601)

© MINDWALKER (downloaded from http://cordis.europa.eu/fetch?CALLER=OFFR_TM_EN&ACTION=D&RCN=10601)

After finding a way to collect the EEG/EMG signals and interpret them, the researchers needed to create the exoskeleton (from the CORDIS news release),

The universities of Delft and Twente in the Netherlands proposed an innovative approach for the design of the exoskeleton and its control. The exoskeletonis designed to be sufficiently robust to bear the weight of a 100 kg adult and powerful enough to recover balance from external causes of instability such as the user’s own torso movements during walking or a gentle push from the back or side. Compared to other exoskeletons developed to date it is relatively light, weighing less than 30 kg without batteries, and, because a final version of the system should be self-powered, it is designed to minimise energy consumption.

The Mindwalker researchers achieved energy efficiency through the use of springs fitted inside the joints that are capable of absorbing and recovering some of the energy otherwise dissipated during walking, and through the development of an efficient strategy for controlling the exoskeleton.

Most exoskeletons are designed to be balanced when stationary or quasi-static and to move by little steps inside their ground stability perimeter, an approach known as ‘Zero moment point’, or ZMP. Although this approach is commonly used for controlling humanoid robots, when applied to exoskeletons, it makes them heavy and slow – and usually requires users to be assisted by a walking frame, sticks or some other support device when they move.

Alternatively, a more advanced and more natural control strategy can replicate the way humans actually walk, with a controlled loss of balance in the walking direction.

‘This approach is called “Limit-cycle walking” and has been implemented using model predictive control to predict the behaviour of the user and exoskeleton and for controlling the exoskeleton during the walk. This was the approach investigated in Mindwalker,’ Mr Ilzkovitz says.

To train users to control the exoskeleton, researchers from Space Applications Services developed a virtual-reality training platform, providing an immersive environment in which new users can safely become accustomed to using the system before testing it out in a clinical setting, and, the team hope, eventually using it in everyday life.

By the end of this year, tests with able-bodied trial users will be completed. The system will then be transferred to the Foundation Santa Lucia for conducting a clinical evaluation until May 2013 with five to 10volunteers suffering from spinal cord injuries. These trials will help identify shortcomings and any areas of performance improvement, the project coordinator says.

In the meantime, the project partners are continuing research on different components for a variety of potential applications. The project coordinator notes, for example, that elements of the system could be adapted for the rehabilitation of stroke victims or to develop easy-to-use exoskeletons for elderly people for mobility support.

Space Applications Services, meanwhile, is also exploring applications of the Mindwalker technology to train astronauts and help them rebuild muscle mass after spending long periods of time in zero-gravity environments.

There’s more about the European Commission’s Seventh Programme-funded Mindwalker project here.

Parallel with these developments in Europe, Miguel Nicolelis of Duke University has stated that he will have a working exoskeleton (Walk Again Project)  for the kickoff by a paraplegic individual for the opening of the World Cup (soccer/football) in Brazil in 2014. I mentioned Nicolelis and his work most recently in a Mar. 4, 2013 posting.

Taken together, this research which strongly suggests that people can perceive prostheses as being part of their bodies and exoskeletons that are powered by the wearer’s thoughts, we seem to be edging closer to a world where machines and humans become one.

Brain-to-brain communication, organic computers, and BAM (brain activity map), the connectome

Miguel Nicolelis, a professor at Duke University, has been making international headlines lately with two brain projects. The first one about implanting a brain chip that allows rats to perceive infrared light was mentioned in my Feb. 15, 2013 posting. The latest project is a brain-to-brain (rats) communication project as per a Feb. 28, 2013 news release on *EurekAlert,

Researchers have electronically linked the brains of pairs of rats for the first time, enabling them to communicate directly to solve simple behavioral puzzles. A further test of this work successfully linked the brains of two animals thousands of miles apart—one in Durham, N.C., and one in Natal, Brazil.

The results of these projects suggest the future potential for linking multiple brains to form what the research team is calling an “organic computer,” which could allow sharing of motor and sensory information among groups of animals. The study was published Feb. 28, 2013, in the journal Scientific Reports.

“Our previous studies with brain-machine interfaces had convinced us that the rat brain was much more plastic than we had previously thought,” said Miguel Nicolelis, M.D., PhD, lead author of the publication and professor of neurobiology at Duke University School of Medicine. “In those experiments, the rat brain was able to adapt easily to accept input from devices outside the body and even learn how to process invisible infrared light generated by an artificial sensor. So, the question we asked was, ‘if the brain could assimilate signals from artificial sensors, could it also assimilate information input from sensors from a different body?'”

Ben Schiller in a Mar. 1, 2013 article for Fast Company describes both the latest experiment and the work leading up to it,

First, two rats were trained to press a lever when a light went on in their cage. Press the right lever, and they would get a reward–a sip of water. The animals were then split in two: one cage had a lever with a light, while another had a lever without a light. When the first rat pressed the lever, the researchers sent electrical activity from its brain to the second rat. It pressed the right lever 70% of the time (more than half).

In another experiment, the rats seemed to collaborate. When the second rat didn’t push the right lever, the first rat was denied a drink. That seemed to encourage the first to improve its signals, raising the second rat’s lever-pushing success rate.

Finally, to show that brain-communication would work at a distance, the researchers put one rat in an cage in North Carolina, and another in Natal, Brazil. Despite noise on the Internet connection, the brain-link worked just as well–the rate at which the second rat pushed the lever was similar to the experiment conducted solely in the U.S.

The Duke University Feb. 28, 2013 news release, the origin for the news release on EurekAlert, provides more specific details about the experiments and the rats’ training,

To test this hypothesis, the researchers first trained pairs of rats to solve a simple problem: to press the correct lever when an indicator light above the lever switched on, which rewarded the rats with a sip of water. They next connected the two animals’ brains via arrays of microelectrodes inserted into the area of the cortex that processes motor information.

One of the two rodents was designated as the “encoder” animal. This animal received a visual cue that showed it which lever to press in exchange for a water reward. Once this “encoder” rat pressed the right lever, a sample of its brain activity that coded its behavioral decision was translated into a pattern of electrical stimulation that was delivered directly into the brain of the second rat, known as the “decoder” animal.

The decoder rat had the same types of levers in its chamber, but it did not receive any visual cue indicating which lever it should press to obtain a reward. Therefore, to press the correct lever and receive the reward it craved, the decoder rat would have to rely on the cue transmitted from the encoder via the brain-to-brain interface.

The researchers then conducted trials to determine how well the decoder animal could decipher the brain input from the encoder rat to choose the correct lever. The decoder rat ultimately achieved a maximum success rate of about 70 percent, only slightly below the possible maximum success rate of 78 percent that the researchers had theorized was achievable based on success rates of sending signals directly to the decoder rat’s brain.

Importantly, the communication provided by this brain-to-brain interface was two-way. For instance, the encoder rat did not receive a full reward if the decoder rat made a wrong choice. The result of this peculiar contingency, said Nicolelis, led to the establishment of a “behavioral collaboration” between the pair of rats.

“We saw that when the decoder rat committed an error, the encoder basically changed both its brain function and behavior to make it easier for its partner to get it right,” Nicolelis said. “The encoder improved the signal-to-noise ratio of its brain activity that represented the decision, so the signal became cleaner and easier to detect. And it made a quicker, cleaner decision to choose the correct lever to press. Invariably, when the encoder made those adaptations, the decoder got the right decision more often, so they both got a better reward.”

In a second set of experiments, the researchers trained pairs of rats to distinguish between a narrow or wide opening using their whiskers. If the opening was narrow, they were taught to nose-poke a water port on the left side of the chamber to receive a reward; for a wide opening, they had to poke a port on the right side.

The researchers then divided the rats into encoders and decoders. The decoders were trained to associate stimulation pulses with the left reward poke as the correct choice, and an absence of pulses with the right reward poke as correct. During trials in which the encoder detected the opening width and transmitted the choice to the decoder, the decoder had a success rate of about 65 percent, significantly above chance.

To test the transmission limits of the brain-to-brain communication, the researchers placed an encoder rat in Brazil, at the Edmond and Lily Safra International Institute of Neuroscience of Natal (ELS-IINN), and transmitted its brain signals over the Internet to a decoder rat in Durham, N.C. They found that the two rats could still work together on the tactile discrimination task.

“So, even though the animals were on different continents, with the resulting noisy transmission and signal delays, they could still communicate,” said Miguel Pais-Vieira, PhD, a postdoctoral fellow and first author of the study. “This tells us that it could be possible to create a workable, network of animal brains distributed in many different locations.”

Will Oremus in his Feb. 28, 2013 article for Slate seems a little less buoyant about the implications of this work,

Nicolelis believes this opens the possibility of building an “organic computer” that links the brains of multiple animals into a single central nervous system, which he calls a “brain-net.” Are you a little creeped out yet? In a statement, Nicolelis adds:

We cannot even predict what kinds of emergent properties would appear when animals begin interacting as part of a brain-net. In theory, you could imagine that a combination of brains could provide solutions that individual brains cannot achieve by themselves.

That sounds far-fetched. But Nicolelis’ lab is developing quite the track record of “taking science fiction and turning it into science,” says Ron Frostig, a neurobiologist at UC-Irvine who was not involved in the rat study. “He’s the most imaginative neuroscientist right now.” (Frostig made it clear he meant this as a complement, though skeptics might interpret the word less charitably.)

The most extensive coverage I’ve given Nicolelis and his work (including the Walk Again project) was in a March 16, 2012 post titled, Monkeys, mind control, robots, prosthetics, and the 2014 World Cup (soccer/football), although there are other mentions including in this Oct. 6, 2011 posting titled, Advertising for the 21st Century: B-Reel, ‘storytelling’, and mind control.  By the way, Nicolelis hopes to have a paraplegic individual (using technology Nicolelis is developing for the Walk Again project) kick the opening soccer/football to the 2014 World Cup games in Brazil.

While there’s much excitement about Nicolelis and his work, there are other ‘brain’ projects being developed in the US including the Brain Activity Map (BAM), which James Lewis notes in his Mar. 1, 2013 posting on the Foresight Institute blog,

A proposal alluded to by President Obama in his State of the Union address [Feb. 2013] to construct a dynamic “functional connectome” Brain Activity Map (BAM) would leverage current progress in neuroscience, synthetic biology, and nanotechnology to develop a map of each firing of every neuron in the human brain—a hundred billion neurons sampled on millisecond time scales. Although not the intended goal of this effort, a project on this scale, if it is funded, should also indirectly advance efforts to develop artificial intelligence and atomically precise manufacturing.

As Lewis notes in his posting, there’s an excellent description of BAM and other brain projects, as well as a discussion about how these ideas are linked (not necessarily by individuals but by the overall direction of work being done in many labs and in many countries across the globe) in Robert Blum’s Feb. (??), 2013 posting titled, BAM: Brain Activity Map Every Spike from Every Neuron, on his eponymous blog. Blum also offers an extensive set of links to the reports and stories about BAM. From Blum’s posting,

The essence of the BAM proposal is to create the technology over the coming decade
to be able to record every spike from every neuron in the brain of a behaving organism.
While this notion seems insanely ambitious, coming from a group of top investigators,
the paper deserves scrutiny. At minimum it shows what might be achieved in the future
by the combination of nanotechnology and neuroscience.

In 2013, as I write this, two European Flagship projects have just received funding for
one billion euro each (1.3 billion dollars each). The Human Brain Project is
an outgrowth of the Blue Brain Project, directed by Prof. Henry Markram
in Lausanne, which seeks to create a detailed simulation of the human brain.
The Graphene Flagship, based in Sweden, will explore uses of graphene for,
among others, creation of nanotech-based supercomputers. The potential synergy
between these projects is a source of great optimism.

The goal of the BAM Project is to elaborate the functional connectome
of a live organism: that is, not only the static (axo-dendritic) connections
but how they function in real-time as thinking and action unfold.

The European Flagship Human Brain Project will create the computational
capability to simulate large, realistic neural networks. But to compare the model
with reality, a real-time, functional, brain-wide connectome must also be created.
Nanotech and neuroscience are mature enough to justify funding this proposal.

I highly recommend reading Blum’s technical description of neural spikes as understanding that concept or any other in his post doesn’t require an advanced degree. Note: Blum holds a number of degrees and diplomas including an MD (neuroscience) from the University of California at San Francisco and a PhD in computer science and biostatistics from California’s Stanford University.

The Human Brain Project has been mentioned here previously. The  most recent mention is in a Jan. 28, 2013 posting about its newly gained status as one of two European Flagship initiatives (the other is the Graphene initiative) each meriting one billion euros of research funding over 10 years. Today, however, is the first time I’ve encountered the BAM project and I’m fascinated. Luckily, John Markoff’s Feb. 17, 2013 article for The New York Times provides some insight into this US initiative (Note: I have removed some links),

The Obama administration is planning a decade-long scientific effort to examine the workings of the human brain and build a comprehensive map of its activity, seeking to do for the brain what the Human Genome Project did for genetics.

The project, which the administration has been looking to unveil as early as March, will include federal agencies, private foundations and teams of neuroscientists and nanoscientists in a concerted effort to advance the knowledge of the brain’s billions of neurons and gain greater insights into perception, actions and, ultimately, consciousness.

Moreover, the project holds the potential of paving the way for advances in artificial intelligence.

What I find particularly interesting is the reference back to the human genome project, which may explain why BAM is also referred to as a ‘connectome’.

ETA Mar.6.13: I have found a Human Connectome Project Mar. 6, 2013 news release on EurekAlert, which leaves me confused. This does not seem to be related to BAM, although the articles about BAM did reference a ‘connectome’. At this point, I’m guessing that BAM and the ‘Human Connectome Project’ are two related but different projects and the reference to a ‘connectome’ in the BAM material is meant generically.  I previously mentioned the Human Connectome Project panel discussion held at the AAAS (American Association for the Advancement of Science) 2013 meeting in my Feb. 7, 2013 posting.

* Corrected EurkAlert to EurekAlert on June 14, 2013.

‘Touching’ infrared light, if you’re a rat followed by announcement of US FDA approval of first commercial artificial retina (bionic eye)

Researcher Miguel Nicolelis and his colleagues at Duke University have implanted a neuroprosthetic device in the portion of a rat’s brain related to touch that allows the rats to see infrared light. From the Feb. 12, 2013 news release on EurekAlert,

Researchers have given rats the ability to “touch” infrared light, normally invisible to them, by fitting them with an infrared detector wired to microscopic electrodes implanted in the part of the mammalian brain that processes tactile information. The achievement represents the first time a brain-machine interface has augmented a sense in adult animals, said Duke University neurobiologist Miguel Nicolelis, who led the research team.

The experiment also demonstrated for the first time that a novel sensory input could be processed by a cortical region specialized in another sense without “hijacking” the function of this brain area said Nicolelis. This discovery suggests, for example, that a person whose visual cortex was damaged could regain sight through a neuroprosthesis implanted in another cortical region, he said.

Although the initial experiments tested only whether rats could detect infrared light, there seems no reason that these animals in the future could not be given full-fledged infrared vision, said Nicolelis. For that matter, cortical neuroprostheses could be developed to give animals or humans the ability to see in any region of the electromagnetic spectrum, or even magnetic fields. “We could create devices sensitive to any physical energy,” he said. “It could be magnetic fields, radio waves, or ultrasound. We chose infrared initially because it didn’t interfere with our electrophysiological recordings.”

Interestingly, the research was supported by the US National Institute of Mental Health (as per the news release).

The researchers have more to say about what they’re doing,

“The philosophy of the field of brain-machine interfaces has until now been to attempt to restore a motor function lost to lesion or damage of the central nervous system,” said Thomson, [Eric Thomson] first author of the study. “This is the first paper in which a neuroprosthetic device was used to augment function—literally enabling a normal animal to acquire a sixth sense.”

Here’s how they conducted the research,

The mammalian retina is blind to infrared light, and mammals cannot detect any heat generated by the weak infrared light used in the studies. In their experiments, the researchers used a test chamber that contained three light sources that could be switched on randomly. Using visible LED lights, they first taught each rat to choose the active light source by poking its nose into an attached port to receive a reward of a sip of water.

After training the rats, the researchers implanted in their brains an array of stimulating microelectrodes, each roughly a tenth the diameter of a human hair. The microelectrodes were implanted in the cortical region that processes touch information from the animals’ facial whiskers.

Attached to the microelectrodes was an infrared detector affixed to the animals’ foreheads. The system was programmed so that orientation toward an infrared light would trigger an electrical signal to the brain. The signal pulses increased in frequency with the intensity and proximity of the light.

The researchers returned the animals to the test chamber, gradually replacing the visible lights with infrared lights. At first in infrared trials, when a light was switched on the animals would tend to poke randomly at the reward ports and scratch at their faces, said Nicolelis. This indicated that they were initially interpreting the brain signals as touch. However, over about a month, the animals learned to associate the brain signal with the infrared source. They began to actively “forage” for the signal, sweeping their heads back and forth to guide themselves to the active light source. Ultimately, they achieved a near-perfect score in tracking and identifying the correct location of the infrared light source.

To ensure that the animals were really using the infrared detector and not their eyes to sense the infrared light, the researchers conducted trials in which the light switched on, but the detector sent no signal to the brain. In these trials, the rats did not react to the infrared light.

Their finding could have an impact on notions of mammalian brain plasticity,

A key finding, said Nicolelis, was that enlisting the touch cortex for light detection did not reduce its ability to process touch signals. “When we recorded signals from the touch cortex of these animals, we found that although the cells had begun responding to infrared light, they continued to respond to whisker touch. It was almost like the cortex was dividing itself evenly so that the neurons could process both types of information.

This finding of brain plasticity is in contrast with the “optogenetic” approach to brain stimulation, which holds that a particular neuronal cell type should be stimulated to generate a desired neurological function. Rather, said Nicolelis, the experiments demonstrate that a broad electrical stimulation, which recruits many distinct cell types, can drive a cortical region to adapt to a new source of sensory input.

All of this work is part of Nicolelis’ larger project ‘Walk Again’ which is mentioned in my March 16, 2012 posting and includes a reference to some ethical issues raised by the work. Briefly, Nicolelis and an international team of collaborators are developing a brain-machine interface that will enable full mobility for people who are severely paralyzed. From the news release,

The Walk Again Project has recently received a $20 million grant from FINEP, a Brazilian research funding agency to allow the development of the first brain-controlled whole body exoskeleton aimed at restoring mobility in severely paralyzed patients. A first demonstration of this technology is expected to happen in the opening game of the 2014 Soccer World Cup in Brazil.

Expanding sensory abilities could also enable a new type of feedback loop to improve the speed and accuracy of such exoskeletons, said Nicolelis. For example, while researchers are now seeking to use tactile feedback to allow patients to feel the movements produced by such “robotic vests,” the feedback could also be in the form of a radio signal or infrared light that would give the person information on the exoskeleton limb’s position and encounter with objects.

There’s more information including videos about the work with infrared light and rats at the Nicolelis Lab website.  Here’s a citation for and link to the team’s research paper,

Perceiving invisible light through a somatosensory cortical prosthesis by Eric E. Thomson, Rafael Carra, & Miguel A.L. Nicolelis. Nature Communications Published 12 Feb 2013 DOI: 10.1038/ncomms2497

Meanwhile, the US Food and Drug Administraton (FDA) has approved the first commercial artificial retina, from the Feb. 14, 2013 news release,

The U.S. Food and Drug Administration (FDA) granted market approval to an artificial retina technology today, the first bionic eye to be approved for patients in the United States. The prosthetic technology was developed in part with support from the National Science Foundation (NSF).

The device, called the Argus® II Retinal Prosthesis System, transmits images from a small, eye-glass-mounted camera wirelessly to a microelectrode array implanted on a patient’s damaged retina. The array sends electrical signals via the optic nerve, and the brain interprets a visual image.

The FDA approval currently applies to individuals who have lost sight as a result of severe to profound retinitis pigmentosa (RP), an ailment that affects one in every 4,000 Americans. The implant allows some individuals with RP, who are completely blind, to locate objects, detect movement, improve orientation and mobility skills and discern shapes such as large letters.

The Argus II is manufactured by, and will be distributed by, Second Sight Medical Products of Sylmar, Calif., which is part of the team of scientists and engineers from the university, federal and private sectors who spent nearly two decades developing the system with public and private investment.

Scientists are often compelled to do research in an area inspired by family,

“Seeing my grandmother go blind motivated me to pursue ophthalmology and biomedical engineering to develop a treatment for patients for whom there was no foreseeable cure,” says the technology’s co-developer, Mark Humayun, associate director of research at the Doheny Eye Institute at the University of Southern California and director of the NSF Engineering Research Center for Biomimetic MicroElectronic Systems (BMES). …”

There’s also been considerable government investment,

The effort by Humayun and his colleagues has received early and continuing support from NSF, the National Institutes of Health and the Department of Energy, with grants totaling more than $100 million. The private sector’s support nearly matched that of the federal government.

“The retinal implant exemplifies how NSF grants for high-risk, fundamental research can directly result in ground-breaking technologies decades later,” said Acting NSF Assistant Director for Engineering Kesh Narayanan. “In collaboration with the Second Sight team and the courageous patients who volunteered to have experimental surgery to implant the first-generation devices, the researchers of NSF’s Biomimetic MicroElectronic Systems Engineering Research Center are developing technologies that may ultimately have as profound an impact on blindness as the cochlear implant has had for hearing loss.”

Leaving aside controversies about cochlear implants and the possibility of such controversies with artificial retinas (bionic eyes), it’s interesting to note that this device is dependent on an external camera,

The researchers’ efforts have bridged cellular biology–necessary for understanding how to stimulate the retinal ganglion cells without permanent damage–with microelectronics, which led to the miniaturized, low-power integrated chip for performing signal conversion, conditioning and stimulation functions. The hardware was paired with software processing and tuning algorithms that convert visual imagery to stimulation signals, and the entire system had to be incorporated within hermetically sealed packaging that allowed the electronics to operate in the vitreous fluid of the eye indefinitely. Finally, the research team had to develop new surgical techniques in order to integrate the device with the body, ensuring accurate placement of the stimulation electrodes on the retina.

“The artificial retina is a great engineering challenge under the interdisciplinary constraint of biology, enabling technology, regulatory compliance, as well as sophisticated design science,” adds Liu.  [Wentai Liu of the University of California, Los Angeles] “The artificial retina provides an interface between biotic and abiotic systems. Its unique design characteristics rely on system-level optimization, rather than the more common practice of component optimization, to achieve miniaturization and integration. Using the most advanced semiconductor technology, the engine for the artificial retina is a ‘system on a chip’ of mixed voltages and mixed analog-digital design, which provides self-contained power and data management and other functionality. This design for the artificial retina facilitates both surgical procedures and regulatory compliance.”

The Argus II design consists of an external video camera system matched to the implanted retinal stimulator, which contains a microelectrode array that spans 20 degrees of visual field. [emphasis mine] …

“The external camera system-built into a pair of glasses-streams video to a belt-worn computer, which converts the video into stimulus commands for the implant,” says Weiland [USC researcher Jim Weiland], “The belt-worn computer encodes the commands into a wireless signal that is transmitted to the implant, which has the necessary electronics to receive and decode both wireless power and data. Based on those data, the implant stimulates the retina with small electrical pulses. The electronics are hermetically packaged and the electrical stimulus is delivered to the retina via a microelectrode array.”

You can see some footage of people using artificial retinas in the context of Grégoire Cosendai’s TEDx Vienna presentation. As I noted in my Aug. 18, 2011 posting where this talk and developments in human enhancement are mentioned, the relevant material can be seen at approximately 13 mins., 25 secs. in Cosendai’s talk.

Second Sight Medical Devices can be found here.

Monkeys, mind control, robots, prosthetics, and the 2014 World Cup (soccer/football)

The idea that a monkey in the US could control a robot’s movements in Japan is stunning. Even more stunning is the fact that the research is four years old. It was discussed publicly in a Jan. 15, 2008 article by Sharon Gaudin for Computer World,

Scientists in the U.S. and Japan have successfully used a monkey’s brain activity to control a humanoid robot — over the Internet.

This research may only be a few years away from helping paralyzed people walk again by enabling them to use their thoughts to control exoskeletons attached to their bodies, according to Miguel Nicolelis, a professor of neurobiology at Duke University and lead researcher on the project.

“This is an attempt to restore mobility to people,” said Nicolelis. “We had the animal trained to walk on a treadmill. As it walked, we recorded its brain activity that generated its locomotion pattern. As the animal was walking and slowing down and changing his pattern, his brain activity was driving a robot in Japan in real time.”

This video clip features an animated monkey simulating control of  a real robot in Japan (the Computational Brain Project of the Japan Science and Technology Agency (JST) in Kyoto partnered with Duke University for this project),

I wonder if the Duke researchers or communications staff thought that the sight of real rhesus monkeys on treadmills might be too disturbing. While we’re on the topic of simulation, I wonder where the robot in the clip actually resides. Quibbles about the video clip aside, I have no doubt that the research took place.

There’s a more recent (Oct. 5, 2011) article, about the work being done in Nicolelis’ laboratory at Duke University, by Ed Yong for Discover Magazine (mentioned previously described in my Oct. 6, 2011 posting),

This is where we are now: at Duke University, a monkey controls a virtual arm using only its thoughts. Miguel Nicolelis had fitted the animal with a headset of electrodes that translates its brain activity into movements. It can grab virtual objects without using its arms. It can also feel the objects without its hands, because the headset stimulates its brain to create the sense of different textures. Monkey think, monkey do, monkey feel – all without moving a muscle.
And this is where  Nicolelis wants to be in three years: a young quadriplegic Brazilian man strolls confidently into a massive stadium. He controls his four prosthetic limbs with his thoughts, and they in turn send tactile information straight to his brain. The technology melds so fluidly with his mind that he confidently runs up and delivers the opening kick of the 2014 World Cup.

This sounds like a far-fetched dream, but Nicolelis – a big soccer fan – is talking to the Brazilian government to make it a reality.

According to Yong, Nicolelis has created an international consortium to support the Walk Again Project. From the project home page,

The Walk Again Project, an international consortium of leading research centers around the world represents a new paradigm for scientific collaboration among the world’s academic institutions, bringing together a global network of scientific and technological experts, distributed among all the continents, to achieve a key humanitarian goal.

The project’s central goal is to develop and implement the first BMI [brain-machine interface] capable of restoring full mobility to patients suffering from a severe degree of paralysis. This lofty goal will be achieved by building a neuroprosthetic device that uses a BMI as its core, allowing the patients to capture and use their own voluntary brain activity to control the movements of a full-body prosthetic device. This “wearable robot,” also known as an “exoskeleton,” will be designed to sustain and carry the patient’s body according to his or her mental will.

In addition to proposing to develop new technologies that aim at improving the quality of life of millions of people worldwide, the Walk Again Project also innovates by creating a complete new paradigm for global scientific collaboration among leading academic institutions worldwide. According to this model, a worldwide network of leading scientific and technological experts, distributed among all the continents, come together to participate in a major, non-profit effort to make a fellow human being walk again, based on their collective expertise. These world renowned scholars will contribute key intellectual assets as well as provide a base for continued fundraising capitalization of the project, setting clear goals to establish fundamental advances toward restoring full mobility for patients in need.

It’s the exoskeleton described on the Walk Again Project home page that Nicolelis is hoping will enable a young Brazilian quadriplegic to deliver the opening kick for the 2014 World Cup (soccer/football) in Brazil.