Tag Archives: Bilge Yildiz

Energy-efficient artificial synapse

This is the second neuromorphic computing chip story from MIT this summer in what has turned out to be a bumper crop of research announcements in this field. The first MIT synapse story was featured in a June 16, 2020 posting. Now, there’s a second and completely different team announcing results for their artificial brain synapse work in a June 19, 2020 news item on Nanowerk (Note: A link has been removed),

Teams around the world are building ever more sophisticated artificial intelligence systems of a type called neural networks, designed in some ways to mimic the wiring of the brain, for carrying out tasks such as computer vision and natural language processing.

Using state-of-the-art semiconductor circuits to simulate neural networks requires large amounts of memory and high power consumption. Now, an MIT [Massachusetts Institute of Technology] team has made strides toward an alternative system, which uses physical, analog devices that can much more efficiently mimic brain processes.

The findings are described in the journal Nature Communications (“Protonic solid-state electrochemical synapse for physical neural networks”), in a paper by MIT professors Bilge Yildiz, Ju Li, and Jesús del Alamo, and nine others at MIT and Brookhaven National Laboratory. The first author of the paper is Xiahui Yao, a former MIT postdoc now working on energy storage at GRU Energy Lab.

That description of the work is one pretty much every team working on developing memristive (neuromorphic) chips could use.

On other fronts, the team has produced a very attractive illustration accompanying this research (aside: Is it my imagination or has there been a serious investment in the colour pink and other pastels for science illustrations?),

A new system developed at MIT and Brookhaven National Lab could provide a faster, more reliable and much more energy efficient approach to physical neural networks, by using analog ionic-electronic devices to mimic synapses.. Courtesy of the researchers

A June 19, 2020 MIT news release, which originated the news item, provides more insight into this specific piece of research (hint: it’s about energy use and repeatability),

Neural networks attempt to simulate the way learning takes place in the brain, which is based on the gradual strengthening or weakening of the connections between neurons, known as synapses. The core component of this physical neural network is the resistive switch, whose electronic conductance can be controlled electrically. This control, or modulation, emulates the strengthening and weakening of synapses in the brain.

In neural networks using conventional silicon microchip technology, the simulation of these synapses is a very energy-intensive process. To improve efficiency and enable more ambitious neural network goals, researchers in recent years have been exploring a number of physical devices that could more directly mimic the way synapses gradually strengthen and weaken during learning and forgetting.

Most candidate analog resistive devices so far for such simulated synapses have either been very inefficient, in terms of energy use, or performed inconsistently from one device to another or one cycle to the next. The new system, the researchers say, overcomes both of these challenges. “We’re addressing not only the energy challenge, but also the repeatability-related challenge that is pervasive in some of the existing concepts out there,” says Yildiz, who is a professor of nuclear science and engineering and of materials science and engineering.

“I think the bottleneck today for building [neural network] applications is energy efficiency. It just takes too much energy to train these systems, particularly for applications on the edge, like autonomous cars,” says del Alamo, who is the Donner Professor in the Department of Electrical Engineering and Computer Science. Many such demanding applications are simply not feasible with today’s technology, he adds.

The resistive switch in this work is an electrochemical device, which is made of tungsten trioxide (WO3) and works in a way similar to the charging and discharging of batteries. Ions, in this case protons, can migrate into or out of the crystalline lattice of the material,  explains Yildiz, depending on the polarity and strength of an applied voltage. These changes remain in place until altered by a reverse applied voltage — just as the strengthening or weakening of synapses does.

The mechanism is similar to the doping of semiconductors,” says Li, who is also a professor of nuclear science and engineering and of materials science and engineering. In that process, the conductivity of silicon can be changed by many orders of magnitude by introducing foreign ions into the silicon lattice. “Traditionally those ions were implanted at the factory,” he says, but with the new device, the ions are pumped in and out of the lattice in a dynamic, ongoing process. The researchers can control how much of the “dopant” ions go in or out by controlling the voltage, and “we’ve demonstrated a very good repeatability and energy efficiency,” he says.

Yildiz adds that this process is “very similar to how the synapses of the biological brain work. There, we’re not working with protons, but with other ions such as calcium, potassium, magnesium, etc., and by moving those ions you actually change the resistance of the synapses, and that is an element of learning.” The process taking place in the tungsten trioxide in their device is similar to the resistance modulation taking place in biological synapses, she says.

“What we have demonstrated here,” Yildiz says, “even though it’s not an optimized device, gets to the order of energy consumption per unit area per unit change in conductance that’s close to that in the brain.” Trying to accomplish the same task with conventional CMOS type semiconductors would take a million times more energy, she says.

The materials used in the demonstration of the new device were chosen for their compatibility with present semiconductor manufacturing systems, according to Li. But they include a polymer material that limits the device’s tolerance for heat, so the team is still searching for other variations of the device’s proton-conducting membrane and better ways of encapsulating its hydrogen source for long-term operations.

“There’s a lot of fundamental research to be done at the materials level for this device,” Yildiz says. Ongoing research will include “work on how to integrate these devices with existing CMOS transistors” adds del Alamo. “All that takes time,” he says, “and it presents tremendous opportunities for innovation, great opportunities for our students to launch their careers.”

Coincidentally or not a University of Massachusetts at Amherst team announced memristor voltage use comparable to human brain voltage use (see my June 15, 2020 posting), plus, there’s a team at Stanford University touting their low-energy biohybrid synapse in a XXX posting. (June 2020 has been a particularly busy month here for ‘artificial brain’ or ‘memristor’ stories.)

Getting back to this latest MIT research, here’s a link to and a citation for the paper,

Protonic solid-state electrochemical synapse for physical neural networks by Xiahui Yao, Konstantin Klyukin, Wenjie Lu, Murat Onen, Seungchan Ryu, Dongha Kim, Nicolas Emond, Iradwikanari Waluyo, Adrian Hunt, Jesús A. del Alamo, Ju Li & Bilge Yildiz. Nature Communications volume 11, Article number: 3134 (2020) DOI: https://doi.org/10.1038/s41467-020-16866-6 Published: 19 June 2020

This paper is open access.

Memristor shakeup

New discoveries suggest that memristors do not function as was previously theorized. (For anyone who wants a memristor description, there’s this Wikipedia entry.) From an Oct. 13, 2015 posting by Alexander Hellemans for the Nanoclast blog (on the IEEE [Institute for Electrical and Electronics Engineers]), Note: Links have been removed,

What’s going to replace flash? The R&D arms of several companies including Hewlett Packard, Intel, and Samsung think the answer might be memristors (also called resistive RAM, ReRAM, or RRAM). These devices have a chance at unseating the non-volatile memory champion because, they use little energy, are very fast, and retain data without requiring power. However, new research indicates that they don’t work in quite the way we thought they do.

The fundamental mechanism at the heart of how a memristor works is something called an “imperfect point contact,” which was predicted in 1971, long before anybody had built working devices. When voltage is applied to a memristor cell, it reduces the resistance across the device. This change in resistance can be read out by applying another, smaller voltage. By inverting the voltage, the resistance of the device is returned to its initial value, that is, the stored information is erased.

Over the last decade researchers have produced two commercially promising types of memristors: electrochemical metallization memory (ECM) cells, and valence change mechanism memory (VCM) cells.

Now international research teams lead by Ilia Valov at the Peter Grünberg Institute in Jülich, Germany, report in Nature Nanotechnology and Advanced Materials that they have identified new processes that erase many of the differences between EMC and VCM cells.

Valov and coworkers in Germany, Japan, Korea, Greece, and the United States started investigating memristors that had a tantalum oxide electrolyte and an active tantalum electrode. “Our studies show that these two types of switching mechanisms in fact can be bridged, and we don’t have a purely oxygen type of switching as was believed, but that also positive [metal] ions, originating from the active electrode, are mobile,” explains Valov.

Here are links to and citations for both papers,

Graphene-Modified Interface Controls Transition from VCM to ECM Switching Modes in Ta/TaOx Based Memristive Devices by Michael Lübben, Panagiotis Karakolis, Vassilios Ioannou-Sougleridis, Pascal Normand, Pangiotis Dimitrakis, & Ilia Valov. Advanced Materials DOI: 10.1002/adma.201502574 First published: 10 September 2015

Nanoscale cation motion in TaOx, HfOx and TiOx memristive systems by Anja Wedig, Michael Luebben, Deok-Yong Cho, Marco Moors, Katharina Skaja, Vikas Rana, Tsuyoshi Hasegawa, Kiran K. Adepalli, Bilge Yildiz, Rainer Waser, & Ilia Valov. Nature Nanotechnology (2015) doi:10.1038/nnano.2015.221 Published online 28 September 2015

Both papers are behind paywalls.