Tag Archives: Forschungszentrum Juelich

Memristors could help AIs overcome ‘catastrophic forgetting’

Leave a reply

A March 20,205 news item on SciencDaily describes a ‘novel’ memristor,

They consume extremely little power and behave similarly to brain cells: so-called memristors. Researchers from Jülich [Forschungszentrum Juelich; Germany], led by Ilia Valov, have now introduced novel memristive components in Nature Communications that offer significant advantages over previous versions: they are more robust, function across a wider voltage range, and can operate in both analog and digital modes. These properties could help address the problem of “catastrophic forgetting,” where artificial neural networks abruptly forget previously learned information.

The problem of “catastrophic forgetting” occurs when deep neural networks are trained for a new task. This is because a new optimization simply overwrites a previous one. The brain does not have this problem because it can apparently adjust the degree of synaptic change; experts are now also talking about a so-called “metaplasticity”. They suspect that it is only through these different degrees of plasticity that our brain can permanently learn new tasks without forgetting old content. The new memristor accomplishes something similar.

“Its unique properties allow the use of different switching modes to control the modulation of the memristor in such a way that stored information is not lost,” says Ilia Valov from the Peter Grünberg Institute (PGI-7) at Forschungszentrum Jülich.

…

A March 20, 2025 Forschungszentrum Juelich press release (also on EurekAlert), which originated the news item, provides context for the work along with more technical details,

Ideal candidates for neuro-inspired devices

Modern computer chips are evolving rapidly. Their development could receive a further boost from memristors—a term derived from memory and resistor. These components are essentially resistors with memory: their electrical resistance changes depending on the applied voltage, and unlike conventional switching elements, their resistance value remains even after the voltage is turned off. This is because memristors can undergo structural changes—for example, due to atoms depositing on the electrodes.

“Memristive elements are considered ideal candidates for learning-capable, neuro-inspired computer components modeled on the brain,” says Ilia Valov.

Despite considerable progress and efforts, the commercialization of the components is progressing slower than expected. This is due in particular to an often high failure rate in production and a short lifespan of the products. In addition, they are sensitive to heat generation or mechanical influences, which can lead to frequent malfunctions during operation. “Basic research is therefore essential to better control nanoscale processes,” says Valov, who has been working in this field of memristors for many years. ”We need new materials and switching mechanisms to reduce the complexity of the systems and increase the range of functionalities.”

It is precisely in this regard that the chemist and materials scientist, together with German and Chinese colleagues, has now been able to report an important success: “We have discovered a fundamentally new electrochemical memristive mechanism that is chemically and electrically more stable,” explains Valov. The development has now been presented in the journal Nature Communications.

A New Mechanism for Memristors

“So far, two main mechanisms have been identified for the functioning of so-called bipolar memristors: ECM and VCM,” explains Valov. ECM stands for ‘Electrochemical Metallization’ and VCM for ‘Valence Change Mechanism’.

  • ECM memristors form a metallic filament between the two electrodes—a tiny “conductive bridge” that alters electrical resistance and dissolves again when the voltage is reversed. The critical parameter here is the energy barrier (resistance) of the electrochemical reaction. This design allows for low switching voltages and fast switching times, but the generated states are variable and relatively short-lived.
     
  • VCM memristors, on the other hand, do not change resistance through the movement of metal ions but rather through the movement of oxygen ions at the interface between the electrode and electrolyte—by modifying the so-called Schottky barrier. This process is comparatively stable but requires high switching voltages.

Each type of memristor has its own advantages and disadvantages. “We therefore considered designing a memristor that combines the benefits of both types,” explains Ilia Valov. Among experts, this was previously thought to be impossible. “Our new memristor is based on a completely different principle: it utilizes a filament made of metal oxides rather than a purely metallic one like ECM,” Valov explains. This filament is formed by the movement of oxygen and tantalum ions and is highly stable—it never fully dissolves. “You can think of it as a filament that always exists to some extent and is only chemically modified,” says Valov.

The novel switching mechanism is therefore very robust. The scientists also refer to it as a filament conductivity modification mechanism (FCM). Components based on this mechanism have several advantages: they are chemically and electrically more stable, more resistant to high temperatures, have a wider voltage window and require lower voltages to produce. As a result, fewer components burn out during the manufacturing process, the reject rate is lower and their lifespan is longer.

Perspective solution for “catastrophic forgetting”

On top of that, the different oxidation states allow the memristor to be operated in a binary and/or analog mode. While binary signals are digital and can only output two states, analog signals are continuous and can take on any intermediate value. This combination of analog and digital behavior is particularly interesting for neuromorphic chips because it can help to overcome the problem of “catastrophic forgetting”: deep neural networks delete what they have learned when they are trained for a new task. This is because a new optimization simply overwrites a previous one.

The brain does not have this problem because it can apparently adjust the degree of synaptic change; experts are now also talking about a so-called “metaplasticity”. They suspect that it is only through these different degrees of plasticity that our brain can permanently learn new tasks without forgetting old content. The new ohmic memristor accomplishes something similar. “Its unique properties allow the use of different switching modes to control the modulation of the memristor in such a way that stored information is not lost,” says Valov.

The researchers have already implemented the new memristive component in a model of an artificial neural network in a simulation. In several image data sets, the system achieved a high level of accuracy in pattern recognition. In the future, the team wants to look for other materials for memristors that might work even better and more stably than the version presented here. “Our results will further advance the development of electronics for ‘computation-in-memory’ applications,” Valov is certain.

Here’s a link to and a citation for the paper,

Electrochemical ohmic memristors for continual learning by Shaochuan Chen, Zhen Yang, Heinrich Hartmann, Astrid Besmehn, Yuchao Yang & Ilia Valov. Nature Communications volume 16, Article number: 2348 (2025) DOI: https://doi.org/10.1038/s41467-025-57543-w Published: 08 March 2025

This paper is open access.

Guide for memristive hardware design

Leave a reply

An August 15 ,2022 news item on ScienceDaily announces a type of guide for memristive hardware design,

They are many times faster than flash memory and require significantly less energy: memristive memory cells could revolutionize the energy efficiency of neuromorphic [brainlike] computers. In these computers, which are modeled on the way the human brain works, memristive cells function like artificial synapses. Numerous groups around the world are working on the use of corresponding neuromorphic circuits — but often with a lack of understanding of how they work and with faulty models. Jülich researchers have now summarized the physical principles and models in a comprehensive review article in the renowned journal Advances in Physics.

…

An August 15, 2022 Forschungszentrum Juelich press release (also on EurekAlert), which originated the news item, describes two papers designed to help researchers better understand and design memristive hardware,

Certain tasks – such as recognizing patterns and language – are performed highly efficiently by a human brain, requiring only about one ten-thousandth of the energy of a conventional, so-called “von Neumann” computer. One of the reasons lies in the structural differences: In a von Neumann architecture, there is a clear separation between memory and processor, which requires constant moving of large amounts of data. This is time and energy consuming – the so-called von Neumann bottleneck. In the brain, the computational operation takes place directly in the data memory and the biological synapses perform the tasks of memory and processor at the same time.

In Jülich, scientists have been working for more than 15 years on special data storage devices and components that can have similar properties to the synapses in the human brain. So-called memristive memory devices, also known as memristors, are considered to be extremely fast, energy-saving and can be miniaturized very well down to the nanometer range. The functioning of memristive cells is based on a very special effect: Their electrical resistance is not constant, but can be changed and reset again by applying an external voltage, theoretically continuously. The change in resistance is controlled by the movement of oxygen ions. If these move out of the semiconducting metal oxide layer, the material becomes more conductive and the electrical resistance drops. This change in resistance can be used to store information.

The processes that can occur in cells are very complex and vary depending on the material system. Three researchers from the Jülich Peter Grünberg Institute – Prof. Regina Dittmann, Dr. Stephan Menzel, and Prof. Rainer Waser – have therefore compiled their research results in a detailed review article, “Nanoionic memristive phenomena in metal oxides: the valence change mechanism”. They explain in detail the various physical and chemical effects in memristors and shed light on the influence of these effects on the switching properties of memristive cells and their reliability.

“If you look at current research activities in the field of neuromorphic memristor circuits, they are often based on empirical approaches to material optimization,” said Rainer Waser, director at the Peter Grünberg Institute. “Our goal with our review article is to give researchers something to work with in order to enable insight-driven material optimization.” The team of authors worked on the approximately 200-page article for ten years and naturally had to keep incorporating advances in knowledge.

“The analogous functioning of memristive cells required for their use as artificial synapses is not the normal case. Usually, there are sudden jumps in resistance, generated by the mutual amplification of ionic motion and Joule heat,” explains Regina Dittmann of the Peter Grünberg Institute. “In our review article, we provide researchers with the necessary understanding of how to change the dynamics of the cells to enable an analog operating mode.”

“You see time and again that groups simulate their memristor circuits with models that don’t take into account high dynamics of the cells at all. These circuits will never work.” said Stephan Menzel, who leads modeling activities at the Peter Grünberg Institute and has developed powerful compact models that are now in the public domain (www.emrl.de/jart.html). “In our review article, we provide the basics that are extremely helpful for a correct use of our compact models.”

Roadmap neuromorphic computing

The “Roadmap of Neuromorphic Computing and Engineering”, which was published in May 2022, shows how neuromorphic computing can help to reduce the enormous energy consumption of IT globally. In it, researchers from the Peter Grünberg Institute (PGI-7), together with leading experts in the field, have compiled the various technological possibilities, computational approaches, learning algorithms and fields of application. 

According to the study, applications in the field of artificial intelligence, such as pattern recognition or speech recognition, are likely to benefit in a special way from the use of neuromorphic hardware. This is because they are based – much more so than classical numerical computing operations – on the shifting of large amounts of data. Memristive cells make it possible to process these gigantic data sets directly in memory without transporting them back and forth between processor and memory. This could reduce the energy efficiency of artificial neural networks by orders of magnitude.

Memristive cells can also be interconnected to form high-density matrices that enable neural networks to learn locally. This so-called edge computing thus shifts computations from the data center to the factory floor, the vehicle, or the home of people in need of care. Thus, monitoring and controlling processes or initiating rescue measures can be done without sending data via a cloud. “This achieves two things at the same time: you save energy, and at the same time, personal data and data relevant to security remain on site,” says Prof. Dittmann, who played a key role in creating the roadmap as editor.

Here’s a link to and a citation for the ‘roadmap’,

2022 roadmap on neuromorphic computing and engineering by Dennis V Christensen, Regina Dittmann, Bernabe Linares-Barranco, Abu Sebastian, Manuel Le Gallo, Andrea Redaelli, Stefan Slesazeck, Thomas Mikolajick, Sabina Spiga, Stephan Menzel, Ilia Valov, Gianluca Milano, Carlo Ricciardi, Shi-Jun Liang, Feng Miao, Mario Lanza, Tyler J Quill, Scott T Keene, Alberto Salleo, Julie Grollier, Danijela Marković, Alice Mizrahi, Peng Yao, J Joshua Yang, Giacomo Indiveri, John Paul Strachan, Suman Datta, Elisa Vianello, Alexandre Valentian, Johannes Feldmann, Xuan Li, Wolfram H P Pernice, Harish Bhaskaran, Steve Furber, Emre Neftci, Franz Scherr, Wolfgang Maass, Srikanth Ramaswamy, Jonathan Tapson, Priyadarshini Panda, Youngeun Kim, Gouhei Tanaka, Simon Thorpe, Chiara Bartolozzi, Thomas A Cleland, Christoph Posch, ShihChii Liu, Gabriella Panuccio, Mufti Mahmud, Arnab Neelim Mazumder, Morteza Hosseini, Tinoosh Mohsenin, Elisa Donati, Silvia Tolu, Roberto Galeazzi, Martin Ejsing Christensen, Sune Holm, Daniele Ielmini and N Pryds. Neuromorphic Computing and Engineering , Volume 2, Number 2 DOI: 10.1088/2634-4386/ac4a83 20 May 2022 • © 2022 The Author(s)

This paper is open access.

Here’s the most recent paper,

Nanoionic memristive phenomena in metal oxides: the valence change mechanism by Regina Dittmann, Stephan Menzel & Rainer Waser. Advances in Physics
Volume 70, 2021 – Issue 2 Pages 155-349 DOI: https://doi.org/10.1080/00018732.2022.2084006 Published online: 06 Aug 2022

This paper is behind a paywall.

New design directions to increase variety, efficiency, selectivity and reliability for memristive devices

Leave a reply

A May 11, 2020 news item on ScienceDaily provides a description of the current ‘memristor scene’ along with an announcement about a piece of recent research,

Scientists around the world are intensively working on memristive devices, which are capable in extremely low power operation and behave similarly to neurons in the brain. Researchers from the Jülich Aachen Research Alliance (JARA) and the German technology group Heraeus have now discovered how to systematically control the functional behaviour of these elements. The smallest differences in material composition are found crucial: differences so small that until now experts had failed to notice them. The researchers’ design directions could help to increase variety, efficiency, selectivity and reliability for memristive technology-based applications, for example for energy-efficient, non-volatile storage devices or neuro-inspired computers.

Memristors are considered a highly promising alternative to conventional nanoelectronic elements in computer Chips [sic]. Because of the advantageous functionalities, their development is being eagerly pursued by many companies and research institutions around the world. The Japanese corporation NEC installed already the first prototypes in space satellites back in 2017. Many other leading companies such as Hewlett Packard, Intel, IBM, and Samsung are working to bring innovative types of computer and storage devices based on memristive elements to market.

Fundamentally, memristors are simply “resistors with memory,” in which high resistance can be switched to low resistance and back again. This means in principle that the devices are adaptive, similar to a synapse in a biological nervous system. “Memristive elements are considered ideal candidates for neuro-inspired computers modelled on the brain, which are attracting a great deal of interest in connection with deep learning and artificial intelligence,” says Dr. Ilia Valov of the Peter Grünberg Institute (PGI-7) at Forschungszentrum Jülich.

In the latest issue of the open access journal Science Advances, he and his team describe how the switching and neuromorphic behaviour of memristive elements can be selectively controlled. According to their findings, the crucial factor is the purity of the switching oxide layer. “Depending on whether you use a material that is 99.999999 % pure, and whether you introduce one foreign atom into ten million atoms of pure material or into one hundred atoms, the properties of the memristive elements vary substantially” says Valov.

A May 11, 2020 Forschungszentrum Juelich press release (also on EurekAlert), which originated the news item, delves into the theme of increasing control over memristive systems,

This effect had so far been overlooked by experts. It can be used very specifically for designing memristive systems, in a similar way to doping semiconductors in information technology. “The introduction of foreign atoms allows us to control the solubility and transport properties of the thin oxide layers,” explains Dr. Christian Neumann of the technology group Heraeus. He has been contributing his materials expertise to the project ever since the initial idea was conceived in 2015.

“In recent years there has been remarkable progress in the development and use of memristive devices, however that progress has often been achieved on a purely empirical basis,” according to Valov. Using the insights that his team has gained, manufacturers could now methodically develop memristive elements selecting the functions they need. The higher the doping concentration, the slower the resistance of the elements changes as the number of incoming voltage pulses increases and decreases, and the more stable the resistance remains. “This means that we have found a way for designing types of artificial synapses with differing excitability,” explains Valov.

Design specification for artificial synapses

The brain’s ability to learn and retain information can largely be attributed to the fact that the connections between neurons are strengthened when they are frequently used. Memristive devices, of which there are different types such as electrochemical metallization cells (ECMs) or valence change memory cells (VCMs), behave similarly. When these components are used, the conductivity increases as the number of incoming voltage pulses increases. The changes can also be reversed by applying voltage pulses of the opposite polarity.

The JARA researchers conducted their systematic experiments on ECMs, which consist of a copper electrode, a platinum electrode, and a layer of silicon dioxide between them. Thanks to the cooperation with Heraeus researchers, the JARA scientists had access to different types of silicon dioxide: one with a purity of 99.999999 % – also called 8N silicon dioxide – and others containing 100 to 10,000 ppm (parts per million) of foreign atoms. The precisely doped glass used in their experiments was specially developed and manufactured by quartz glass specialist Heraeus Conamic, which also holds the patent for the procedure. Copper and protons acted as mobile doping agents, while aluminium and gallium were used as non-volatile doping.

Synapses, the connections between neurons, have the ability to transmit signals with varying degrees of strength when they are excited by a quick succession of electrical impulses. One effect of this repeated activity is to increase the concentration of calcium ions, with the result that more neurotransmitters are emitted. Depending on the activity, other effects cause long-term structural changes, which impact the strength of the transmission for several hours, or potentially even for the rest of the person’s life. Memristive elements allow the strength of the electrical transmission to be changed in a similar way to synaptic connections, by applying a voltage. In electrochemical metallization cells (ECMs), a metallic filament develops between the two metal electrodes, thus increasing conductivity. Applying voltage pulses with reversed polarity causes the filament to shrink again until the cell reaches its initial high resistance state. Copyright: Forschungszentrum Jülich / Tobias Schlößer

Record switching time confirms theory

Based on their series of experiments, the researchers were able to show that the ECMs’ switching times change as the amount of doping atoms changes. If the switching layer is made of 8N silicon dioxide, the memristive component switches in only 1.4 nanoseconds. To date, the fastest value ever measured for ECMs had been around 10 nanoseconds. By doping the oxide layer of the components with up to 10,000 ppm of foreign atoms, the switching time was prolonged into the range of milliseconds. “We can also theoretically explain our results. This is helping us to understand the physico-chemical processes on the nanoscale and apply this knowledge in the practice” says Valov. Based on generally applicable theoretical considerations, supported by experimental results, some also documented in the literature, he is convinced that the doping/impurity effect occurs and can be employed in all types memristive elements.

Top: In memristive elements (ECMs) with an undoped, high-purity switching layer of silicon oxide (SiO2), copper ions can move very fast. A filament of copper atoms forms correspondingly fast on the platinum electrode. This increases the total device conductivity respectively the capacity. Due to the high mobility of the ions, however, this filament is unstable at low forming voltages. Center: Gallium ions (Ga3+), which are introduced into the cell (non-volatile doping), bind copper ions (Cu2+) in the switching layer. The movement of the ions slows down, leading to lower switching times, but the filament, once formed remains longer stable. Bottom: Doping with aluminium ions (Al3+) slows down the process even more, since aluminium ions bind copper ions even stronger than gallium ions. Filament growth is even slower, while at the same time the stability of the filament is further increased. Depending on the chemical properties of the introduced doping elements, memristive cells – the artificial synapses – can be created with tailor-made switching and neuromorphic properties. Copyright: Forschungszentrum Jülich / Tobias Schloesser

Here’s a link to and a citation for the paper,

Design of defect-chemical properties and device performance in memristive systems by M. Lübben, F. Cüppers, J. Mohr, M. von Witzleben, U. Breuer, R. Waser, C. Neumann, and I. Valov. Science Advances 08 May 2020: Vol. 6, no. 19, eaaz9079 DOI: 10.1126/sciadv.aaz9079

This paper is open access.

For anyone curious about the German technology group, Heraeus, there’s a fascinating history in its Wikipedia entry. The technology company was formally founded in 1851 but it can be traced back to the 17th century and the founding family’s apothecary.