Tag Archives: Gregoire Cosendai

‘Touching’ infrared light, if you’re a rat followed by announcement of US FDA approval of first commercial artificial retina (bionic eye)

Researcher Miguel Nicolelis and his colleagues at Duke University have implanted a neuroprosthetic device in the portion of a rat’s brain related to touch that allows the rats to see infrared light. From the Feb. 12, 2013 news release on EurekAlert,

Researchers have given rats the ability to “touch” infrared light, normally invisible to them, by fitting them with an infrared detector wired to microscopic electrodes implanted in the part of the mammalian brain that processes tactile information. The achievement represents the first time a brain-machine interface has augmented a sense in adult animals, said Duke University neurobiologist Miguel Nicolelis, who led the research team.

The experiment also demonstrated for the first time that a novel sensory input could be processed by a cortical region specialized in another sense without “hijacking” the function of this brain area said Nicolelis. This discovery suggests, for example, that a person whose visual cortex was damaged could regain sight through a neuroprosthesis implanted in another cortical region, he said.

Although the initial experiments tested only whether rats could detect infrared light, there seems no reason that these animals in the future could not be given full-fledged infrared vision, said Nicolelis. For that matter, cortical neuroprostheses could be developed to give animals or humans the ability to see in any region of the electromagnetic spectrum, or even magnetic fields. “We could create devices sensitive to any physical energy,” he said. “It could be magnetic fields, radio waves, or ultrasound. We chose infrared initially because it didn’t interfere with our electrophysiological recordings.”

Interestingly, the research was supported by the US National Institute of Mental Health (as per the news release).

The researchers have more to say about what they’re doing,

“The philosophy of the field of brain-machine interfaces has until now been to attempt to restore a motor function lost to lesion or damage of the central nervous system,” said Thomson, [Eric Thomson] first author of the study. “This is the first paper in which a neuroprosthetic device was used to augment function—literally enabling a normal animal to acquire a sixth sense.”

Here’s how they conducted the research,

The mammalian retina is blind to infrared light, and mammals cannot detect any heat generated by the weak infrared light used in the studies. In their experiments, the researchers used a test chamber that contained three light sources that could be switched on randomly. Using visible LED lights, they first taught each rat to choose the active light source by poking its nose into an attached port to receive a reward of a sip of water.

After training the rats, the researchers implanted in their brains an array of stimulating microelectrodes, each roughly a tenth the diameter of a human hair. The microelectrodes were implanted in the cortical region that processes touch information from the animals’ facial whiskers.

Attached to the microelectrodes was an infrared detector affixed to the animals’ foreheads. The system was programmed so that orientation toward an infrared light would trigger an electrical signal to the brain. The signal pulses increased in frequency with the intensity and proximity of the light.

The researchers returned the animals to the test chamber, gradually replacing the visible lights with infrared lights. At first in infrared trials, when a light was switched on the animals would tend to poke randomly at the reward ports and scratch at their faces, said Nicolelis. This indicated that they were initially interpreting the brain signals as touch. However, over about a month, the animals learned to associate the brain signal with the infrared source. They began to actively “forage” for the signal, sweeping their heads back and forth to guide themselves to the active light source. Ultimately, they achieved a near-perfect score in tracking and identifying the correct location of the infrared light source.

To ensure that the animals were really using the infrared detector and not their eyes to sense the infrared light, the researchers conducted trials in which the light switched on, but the detector sent no signal to the brain. In these trials, the rats did not react to the infrared light.

Their finding could have an impact on notions of mammalian brain plasticity,

A key finding, said Nicolelis, was that enlisting the touch cortex for light detection did not reduce its ability to process touch signals. “When we recorded signals from the touch cortex of these animals, we found that although the cells had begun responding to infrared light, they continued to respond to whisker touch. It was almost like the cortex was dividing itself evenly so that the neurons could process both types of information.

This finding of brain plasticity is in contrast with the “optogenetic” approach to brain stimulation, which holds that a particular neuronal cell type should be stimulated to generate a desired neurological function. Rather, said Nicolelis, the experiments demonstrate that a broad electrical stimulation, which recruits many distinct cell types, can drive a cortical region to adapt to a new source of sensory input.

All of this work is part of Nicolelis’ larger project ‘Walk Again’ which is mentioned in my March 16, 2012 posting and includes a reference to some ethical issues raised by the work. Briefly, Nicolelis and an international team of collaborators are developing a brain-machine interface that will enable full mobility for people who are severely paralyzed. From the news release,

The Walk Again Project has recently received a $20 million grant from FINEP, a Brazilian research funding agency to allow the development of the first brain-controlled whole body exoskeleton aimed at restoring mobility in severely paralyzed patients. A first demonstration of this technology is expected to happen in the opening game of the 2014 Soccer World Cup in Brazil.

Expanding sensory abilities could also enable a new type of feedback loop to improve the speed and accuracy of such exoskeletons, said Nicolelis. For example, while researchers are now seeking to use tactile feedback to allow patients to feel the movements produced by such “robotic vests,” the feedback could also be in the form of a radio signal or infrared light that would give the person information on the exoskeleton limb’s position and encounter with objects.

There’s more information including videos about the work with infrared light and rats at the Nicolelis Lab website.  Here’s a citation for and link to the team’s research paper,

Perceiving invisible light through a somatosensory cortical prosthesis by Eric E. Thomson, Rafael Carra, & Miguel A.L. Nicolelis. Nature Communications Published 12 Feb 2013 DOI: 10.1038/ncomms2497

Meanwhile, the US Food and Drug Administraton (FDA) has approved the first commercial artificial retina, from the Feb. 14, 2013 news release,

The U.S. Food and Drug Administration (FDA) granted market approval to an artificial retina technology today, the first bionic eye to be approved for patients in the United States. The prosthetic technology was developed in part with support from the National Science Foundation (NSF).

The device, called the Argus® II Retinal Prosthesis System, transmits images from a small, eye-glass-mounted camera wirelessly to a microelectrode array implanted on a patient’s damaged retina. The array sends electrical signals via the optic nerve, and the brain interprets a visual image.

The FDA approval currently applies to individuals who have lost sight as a result of severe to profound retinitis pigmentosa (RP), an ailment that affects one in every 4,000 Americans. The implant allows some individuals with RP, who are completely blind, to locate objects, detect movement, improve orientation and mobility skills and discern shapes such as large letters.

The Argus II is manufactured by, and will be distributed by, Second Sight Medical Products of Sylmar, Calif., which is part of the team of scientists and engineers from the university, federal and private sectors who spent nearly two decades developing the system with public and private investment.

Scientists are often compelled to do research in an area inspired by family,

“Seeing my grandmother go blind motivated me to pursue ophthalmology and biomedical engineering to develop a treatment for patients for whom there was no foreseeable cure,” says the technology’s co-developer, Mark Humayun, associate director of research at the Doheny Eye Institute at the University of Southern California and director of the NSF Engineering Research Center for Biomimetic MicroElectronic Systems (BMES). …”

There’s also been considerable government investment,

The effort by Humayun and his colleagues has received early and continuing support from NSF, the National Institutes of Health and the Department of Energy, with grants totaling more than $100 million. The private sector’s support nearly matched that of the federal government.

“The retinal implant exemplifies how NSF grants for high-risk, fundamental research can directly result in ground-breaking technologies decades later,” said Acting NSF Assistant Director for Engineering Kesh Narayanan. “In collaboration with the Second Sight team and the courageous patients who volunteered to have experimental surgery to implant the first-generation devices, the researchers of NSF’s Biomimetic MicroElectronic Systems Engineering Research Center are developing technologies that may ultimately have as profound an impact on blindness as the cochlear implant has had for hearing loss.”

Leaving aside controversies about cochlear implants and the possibility of such controversies with artificial retinas (bionic eyes), it’s interesting to note that this device is dependent on an external camera,

The researchers’ efforts have bridged cellular biology–necessary for understanding how to stimulate the retinal ganglion cells without permanent damage–with microelectronics, which led to the miniaturized, low-power integrated chip for performing signal conversion, conditioning and stimulation functions. The hardware was paired with software processing and tuning algorithms that convert visual imagery to stimulation signals, and the entire system had to be incorporated within hermetically sealed packaging that allowed the electronics to operate in the vitreous fluid of the eye indefinitely. Finally, the research team had to develop new surgical techniques in order to integrate the device with the body, ensuring accurate placement of the stimulation electrodes on the retina.

“The artificial retina is a great engineering challenge under the interdisciplinary constraint of biology, enabling technology, regulatory compliance, as well as sophisticated design science,” adds Liu.  [Wentai Liu of the University of California, Los Angeles] “The artificial retina provides an interface between biotic and abiotic systems. Its unique design characteristics rely on system-level optimization, rather than the more common practice of component optimization, to achieve miniaturization and integration. Using the most advanced semiconductor technology, the engine for the artificial retina is a ‘system on a chip’ of mixed voltages and mixed analog-digital design, which provides self-contained power and data management and other functionality. This design for the artificial retina facilitates both surgical procedures and regulatory compliance.”

The Argus II design consists of an external video camera system matched to the implanted retinal stimulator, which contains a microelectrode array that spans 20 degrees of visual field. [emphasis mine] …

“The external camera system-built into a pair of glasses-streams video to a belt-worn computer, which converts the video into stimulus commands for the implant,” says Weiland [USC researcher Jim Weiland], “The belt-worn computer encodes the commands into a wireless signal that is transmitted to the implant, which has the necessary electronics to receive and decode both wireless power and data. Based on those data, the implant stimulates the retina with small electrical pulses. The electronics are hermetically packaged and the electrical stimulus is delivered to the retina via a microelectrode array.”

You can see some footage of people using artificial retinas in the context of Grégoire Cosendai’s TEDx Vienna presentation. As I noted in my Aug. 18, 2011 posting where this talk and developments in human enhancement are mentioned, the relevant material can be seen at approximately 13 mins., 25 secs. in Cosendai’s talk.

Second Sight Medical Devices can be found here.

God from the machine: Deus ex machina and augmentation

Wherever you go, there it is: ancient Greece. Deus Ex, a game series from Eidos Montréal, is likely referencing ‘deus ex machina’, a term applied to a theatrical device (in both senses of the word) attributed to  playwrights of ancient Greece. (For anyone who’s unfamiliar with the term, at the end of a play, all of the conflicts would be resolved by a god descending from the heavens. The term refers both to the plot device itself and to the mechanical device used to lower the ‘god’.)

The latest game in the series, Deus Ex: Human Revolution, a role-playing shooter, will be released August 23, 2011. From the August 16, 2011 article by Susan Karlin for Fast Company,

The result—Deus Ex: Human Revolution, a role-playing shooter that comes out August 23–extrapolates MicroTransponder, prosthetics, robotics, and other current augmentation technology into a vision of how technologically enhanced people might gain superhuman abilities and at what cost.

… “We built a timeline that traces the history of augmentation, creating new things, and predicting how would it get out into society. We wanted to ground it in today, and make something where everyone could say, ‘I can see the world going that way.'” [Mary DeMarle, Human Revolution’s lead writer]

Human Revolution, although the third in the series, is a prequel to the original Deus Ex which took place 25 years after Human Revolution.

I’m glad to see games that bring up interesting philosophical questions and possible social impacts of emerging technologies along with the action. In a February 3, 2011 interview with Mary DeMarle, Quintin Smith of Rock, Paper, Shotgun posed these questions,

RPS: Finally, with anti-augmentation groups featuring in Human Revolution, I was just wondering what your own opinions are on human augmentation and human bioengineering are.

MD: Oh, gosh. Well I have to tell you that the joke on the team is that for the duration of this story I’d be supporting the anti-technology view, because most people on the team wouldn’t be anti-technology, and it’d help me make the game more human, you know? And now that the project’s over I bought my first iPad, and I have to admit I’m suddenly like “You know, if I could get one of those InfoLinks in my head, it’d be really useful.”

But you know, all of this stuff is already out there. We already have people putting cameras in their eyes to improve their vision. [emphasis mine] The technology’s there, we’re just not aware of it. As far as our team’s technology expert is concerned, human augmentation’s been going on for decades. If you look at all the sports controversy regarding drugs, that is augmentation. It’s already happening.

RPS: But you have no qualms with our using technology to make ourselves more than we can be?

MD: From my perspective, I think mankind will always try to be more than he is. That’s part of being human. But I do admit we have to be careful about how we do it.

In my February 2, 2010 posting (scroll down about 1/2 way), I featured a quote that resonates with DeMarle’s comments about humans trying to be more,

“I don’t think I would have said this if it had never happened,” says Bailey, referring to the accident that tore off his pinkie, ring, and middle fingers. “But I told Touch Bionics I’d cut the rest of my hand off if I could make all five of my fingers robotic.”

Bailey went on to say that having machinery incorporated into his body made him feel “above human”.

As for cameras being implanted in eyes to improve vision, I would be delighted to hear from anyone who has information about this. The only project I could find in my search was EyeBorg, a project with a one-eyed Canadian filmmaker who was planning to have a video camera implanted into his eye socket to record images. From the About the Project page,

Take a one eyed film maker, an unemployed engineer, and a vision for something that’s never been done before and you have yourself the EyeBorg Project. Rob Spence and Kosta Grammatis are trying to make history by embedding a video camera and a transmitter in a prosthetic eye. That eye is going in Robs eye socket, and will record the world from a perspective that’s never been seen before.

There are more details about the EyeBorg project in a June 11, 2010 posting by Tim Hornyak for the Automaton blog (on the IEEE [Institute of Electrical and Electronics Engineers] website),

When Canadian filmmaker Rob Spence was a kid, he would peer through the bionic eye of his Six Million Dollar action figure. After a shooting accident left him partially blind, he decided to create his own electronic eye. Now he calls himself Eyeborg.

Spence’s bionic eye contains a battery-powered, wireless video camera. Not only can he record everything he sees just by looking around, but soon people will be able to log on to his video feed and view the world through his right eye.

I don’t know how the Eyeborg project is proceeding as there haven’t been any updates on the site since August 25, 2010.

While I wish Quintin Smith had asked for more details about the science information DeMarle was passing on in the February 3, 2011 interview, I think it’s interesting to note that information about science and technology comes to us in many ways: advertisements, popular television programmes, comic books, interviews, and games, as well as, formal public science outreach programmes through museums and educational institutions.

ETA August 19, 2011: I found some information about visual prosthetics at the European Commission’s Future and Emerging Technologies (FET) website, We can rebuild you page featuring a TEDxVienna November 2010 talk by electrical engineer, Grégoire Cosendai, from the Swiss Federal Institute of Technology. He doesn’t mention the prosthetics until approximately 13 minutes, 25 seconds into the talk. The work is being done to help people with retinitis pigmentosa, a condition that is incurable at this time but it may have implications for others. There are 30 people worldwide in a clinical trial testing a retinal implant that requires the person wear special glasses containing a camera and an antenna. For Star Trek fans, this seems similar to Geordi LaForge‘s special glasses.

ETA Sept. 13, 2011: Better late than never, here’s an excerpt from Dexter Johnson’s Sept. 2, 2011 posting (on his Nanoclast blog at the Institute of Electrical and Electronics Engineers [IEEE] website) about a nano retina project,

The Israel-based company [Nano Retina] is a joint venture between Rainbow Medical and Zyvex Labs, the latter being well known for its work in nanotechnology and its founder Jim Von Ehr, who has been a strong proponent of molecular mechanosynthesis.

It’s well worth contrasting the information in the company video that Dexter provides and the information in the FET video mentioned in the Aug. 19, 2011 update preceding this one. The company presents a vastly more optimistic claim for the vision these implants will provide than one would expect after viewing the information in the FET video about clinical trials, for another similar (to me) system, currently taking place.