Tag Archives: Giacomo Indiveri

A nontraditional artificial synaptic device and roadmap for Chinese research into neuromorphic devices

Leave a reply

A November 9, 2022 Science China Press press release on EurekAlert announces a new approach to developing neuromorphic (brainlike) devices,

Neuromorphic computing is an information processing model that simulates the efficiency of the human brain with multifunctionality and flexibility. Currently, artificial synaptic devices represented by memristors have been extensively used in neural morphological computing, and different types of neural networks have been developed. However, it is time-consuming and laborious to perform fixing and redeploying of weights stored by traditional artificial synaptic devices. Moreover, synaptic strength is primarily reconstructed via software programming and changing the pulse time, which can result in low efficiency and high energy consumption in neural morphology computing applications.

In a novel research article published in the Beijing-based National Science Review, Prof. Lili Wang from the Chinese Academy of Sciences and her colleagues present a novel hardware neural network based on a tunable flexible MXene energy storage (FMES) system. The system comprises flexible postsynaptic electrodes and MXene nanosheets, which are connected with the presynaptic electrodes using electrolytes. The potential changes in the ion migration process and adsorption in the supercapacitor can simulate information transmission in the synaptic gap. Additionally, the voltage of the FMES system represents the synaptic weight of the connection between two neurons.

Researchers explored the changes of paired-pulse facilitation under different resistance levels to investigate the effect of resistance on the advanced learning and memory behavior of the artificial synaptic system of FMES. The results revealed that the larger the standard deviation, the stronger the memory capacity of the system. In other words, with the continuous improvement of electrical resistance and stimulation time, the memory capacity of the artificial synaptic system of FMES is gradually improved. Therefore, the system can effectively control the accumulation and dissipation of ions by regulating the resistance value in the system without changing the external stimulus, which is expected to realize the coupling of sensing signals and storage weight.

The FMES system can be used to develop neural networks and realize various neural morphological computing tasks, making the recognition accuracy of handwritten digit sets reach 95%. Additionally, the FMES system can simulate the adaptivity of the human brain to achieve adaptive recognition of similar target data sets. Following the training process, the adaptive recognition accuracy can reach approximately 80%, and avoid the time and energy loss caused by recalculation.

“In the future, based on this research, different types of sensors can be integrated on the chip to further realize multimodal sensing computing integrated architecture.” Prof. Lili Wang stated, “The device can perform low-energy calculations, and is expected to solve the problems of high write noise, nonlinear difference, and diffusion under zero bias voltage in certain neural morphological systems.”

Here’s a link to and a citation for the paper,

Neuromorphic-computing-based adaptive learning using ion dynamics in flexible energy storage devices by Shufang Zhao, Wenhao Ran, Zheng Lou, Linlin Li, Swapnadeep Poddar, Lili Wang, Zhiyong Fan, Guozhen Shen. National Science Review, Volume 9, Issue 11, November 2022, nwac158, EOI: https://doi.org/10.1093/nsr/nwac158 Published: 13 August 2022

This paper is open access.

The future (or roadmap for) of Chinese research on neuromorphic engineering

While I was trying (unsuccessfully) to find a copy of the press release on the issuing agency’s website, I found this paper,

2022 roadmap on neuromorphic devices & applications research in China by Qing Wan, Changjin Wan, Huaqiang Wu, Yuchao Yang, Xiaohe Huang, Peng Zhou, LinChen, Tian-Yu Wang, Yi Li, Kanhao Xue, Yuhui He, Xiangshui Miao, Xi Li, Chenchen Xie, Houpeng Chen, Z. T. Song, Hong Wang, Yue Hao, Junyao Zhang, Jia Huang, Zheng Yu Ren, Li Qiang Zhu, Jianyu Du, Chen Ge, Yang Liu, Guanglong Ding, Ye Zhou, Su-Ting Han, Guosheng Wang, Xiao Yu, Bing Chen, Zhufei Chu, Lunyao Wang, Yinshui Xia, Chen Mu, Feng Lin, Chixiao Chen, Bojun Cheng, Yannan Xing, Weitao Zeng, Hong Chen, Lei Yu, Giacomo Indiveri and Ning Qiao. Neuromorphic Computing and Engineering DOI: 10.1088/2634-4386/ac7a5a *Accepted Manuscript online 20 June 2022 • © 2022 The Author(s). Published by IOP Publishing Ltd

The paper is open access.

*From the IOP’s Definitions of article versions: Accepted Manuscript is ‘the version of the article accepted for publication including all changes made as a result of the peer review process, and which may also include the addition to the article by IOP of a header, an article ID, a cover sheet and/or an ‘Accepted Manuscript’ watermark, but excluding any other editing, typesetting or other changes made by IOP and/or its licensors’.*

This is neither the published version nor the version of record.

Guide for memristive hardware design

Leave a reply

An August 15 ,2022 news item on ScienceDaily announces a type of guide for memristive hardware design,

They are many times faster than flash memory and require significantly less energy: memristive memory cells could revolutionize the energy efficiency of neuromorphic [brainlike] computers. In these computers, which are modeled on the way the human brain works, memristive cells function like artificial synapses. Numerous groups around the world are working on the use of corresponding neuromorphic circuits — but often with a lack of understanding of how they work and with faulty models. Jülich researchers have now summarized the physical principles and models in a comprehensive review article in the renowned journal Advances in Physics.

…

An August 15, 2022 Forschungszentrum Juelich press release (also on EurekAlert), which originated the news item, describes two papers designed to help researchers better understand and design memristive hardware,

Certain tasks – such as recognizing patterns and language – are performed highly efficiently by a human brain, requiring only about one ten-thousandth of the energy of a conventional, so-called “von Neumann” computer. One of the reasons lies in the structural differences: In a von Neumann architecture, there is a clear separation between memory and processor, which requires constant moving of large amounts of data. This is time and energy consuming – the so-called von Neumann bottleneck. In the brain, the computational operation takes place directly in the data memory and the biological synapses perform the tasks of memory and processor at the same time.

In Jülich, scientists have been working for more than 15 years on special data storage devices and components that can have similar properties to the synapses in the human brain. So-called memristive memory devices, also known as memristors, are considered to be extremely fast, energy-saving and can be miniaturized very well down to the nanometer range. The functioning of memristive cells is based on a very special effect: Their electrical resistance is not constant, but can be changed and reset again by applying an external voltage, theoretically continuously. The change in resistance is controlled by the movement of oxygen ions. If these move out of the semiconducting metal oxide layer, the material becomes more conductive and the electrical resistance drops. This change in resistance can be used to store information.

The processes that can occur in cells are very complex and vary depending on the material system. Three researchers from the Jülich Peter Grünberg Institute – Prof. Regina Dittmann, Dr. Stephan Menzel, and Prof. Rainer Waser – have therefore compiled their research results in a detailed review article, “Nanoionic memristive phenomena in metal oxides: the valence change mechanism”. They explain in detail the various physical and chemical effects in memristors and shed light on the influence of these effects on the switching properties of memristive cells and their reliability.

“If you look at current research activities in the field of neuromorphic memristor circuits, they are often based on empirical approaches to material optimization,” said Rainer Waser, director at the Peter Grünberg Institute. “Our goal with our review article is to give researchers something to work with in order to enable insight-driven material optimization.” The team of authors worked on the approximately 200-page article for ten years and naturally had to keep incorporating advances in knowledge.

“The analogous functioning of memristive cells required for their use as artificial synapses is not the normal case. Usually, there are sudden jumps in resistance, generated by the mutual amplification of ionic motion and Joule heat,” explains Regina Dittmann of the Peter Grünberg Institute. “In our review article, we provide researchers with the necessary understanding of how to change the dynamics of the cells to enable an analog operating mode.”

“You see time and again that groups simulate their memristor circuits with models that don’t take into account high dynamics of the cells at all. These circuits will never work.” said Stephan Menzel, who leads modeling activities at the Peter Grünberg Institute and has developed powerful compact models that are now in the public domain (www.emrl.de/jart.html). “In our review article, we provide the basics that are extremely helpful for a correct use of our compact models.”

Roadmap neuromorphic computing

The “Roadmap of Neuromorphic Computing and Engineering”, which was published in May 2022, shows how neuromorphic computing can help to reduce the enormous energy consumption of IT globally. In it, researchers from the Peter Grünberg Institute (PGI-7), together with leading experts in the field, have compiled the various technological possibilities, computational approaches, learning algorithms and fields of application. 

According to the study, applications in the field of artificial intelligence, such as pattern recognition or speech recognition, are likely to benefit in a special way from the use of neuromorphic hardware. This is because they are based – much more so than classical numerical computing operations – on the shifting of large amounts of data. Memristive cells make it possible to process these gigantic data sets directly in memory without transporting them back and forth between processor and memory. This could reduce the energy efficiency of artificial neural networks by orders of magnitude.

Memristive cells can also be interconnected to form high-density matrices that enable neural networks to learn locally. This so-called edge computing thus shifts computations from the data center to the factory floor, the vehicle, or the home of people in need of care. Thus, monitoring and controlling processes or initiating rescue measures can be done without sending data via a cloud. “This achieves two things at the same time: you save energy, and at the same time, personal data and data relevant to security remain on site,” says Prof. Dittmann, who played a key role in creating the roadmap as editor.

Here’s a link to and a citation for the ‘roadmap’,

2022 roadmap on neuromorphic computing and engineering by Dennis V Christensen, Regina Dittmann, Bernabe Linares-Barranco, Abu Sebastian, Manuel Le Gallo, Andrea Redaelli, Stefan Slesazeck, Thomas Mikolajick, Sabina Spiga, Stephan Menzel, Ilia Valov, Gianluca Milano, Carlo Ricciardi, Shi-Jun Liang, Feng Miao, Mario Lanza, Tyler J Quill, Scott T Keene, Alberto Salleo, Julie Grollier, Danijela Marković, Alice Mizrahi, Peng Yao, J Joshua Yang, Giacomo Indiveri, John Paul Strachan, Suman Datta, Elisa Vianello, Alexandre Valentian, Johannes Feldmann, Xuan Li, Wolfram H P Pernice, Harish Bhaskaran, Steve Furber, Emre Neftci, Franz Scherr, Wolfgang Maass, Srikanth Ramaswamy, Jonathan Tapson, Priyadarshini Panda, Youngeun Kim, Gouhei Tanaka, Simon Thorpe, Chiara Bartolozzi, Thomas A Cleland, Christoph Posch, ShihChii Liu, Gabriella Panuccio, Mufti Mahmud, Arnab Neelim Mazumder, Morteza Hosseini, Tinoosh Mohsenin, Elisa Donati, Silvia Tolu, Roberto Galeazzi, Martin Ejsing Christensen, Sune Holm, Daniele Ielmini and N Pryds. Neuromorphic Computing and Engineering , Volume 2, Number 2 DOI: 10.1088/2634-4386/ac4a83 20 May 2022 • © 2022 The Author(s)

This paper is open access.

Here’s the most recent paper,

Nanoionic memristive phenomena in metal oxides: the valence change mechanism by Regina Dittmann, Stephan Menzel & Rainer Waser. Advances in Physics
Volume 70, 2021 – Issue 2 Pages 155-349 DOI: https://doi.org/10.1080/00018732.2022.2084006 Published online: 06 Aug 2022

This paper is behind a paywall.

Swiss researchers, memristors, perovskite crystals, and neuromorphic (brainlike) computing

Leave a reply

A May 18, 2022 news item on Nanowerk highlights research into making memristors more ‘flexible’, (Note: There’s an almost identical May 18, 2022 news item on ScienceDaily but the issuing agency is listed as ETH Zurich rather than Empa as listed on Nanowerk),

Compared with computers, the human brain is incredibly energy-efficient. Scientists are therefore drawing on how the brain and its interconnected neurons function for inspiration in designing innovative computing technologies. They foresee that these brain-inspired computing systems, will be more energy-efficient than conventional ones, as well as better at performing machine-learning tasks.

Much like neurons, which are responsible for both data storage and data processing in the brain, scientists want to combine storage and processing in a single type of electronic component, known as a memristor. Their hope is that this will help to achieve greater efficiency because moving data between the processor and the storage, as conventional computers do, is the main reason for the high energy consumption in machine-learning applications.

Researchers at ETH Zurich, Empa and the University of Zurich have now developed an innovative concept for a memristor that can be used in a far wider range of applications than existing memristors.

“There are different operation modes for memristors, and it is advantageous to be able to use all these modes depending on an artificial neural network’s architecture,” explains ETH Zurich postdoc Rohit John. “But previous conventional memristors had to be configured for one of these modes in advance.”

The new memristors can now easily switch between two operation modes while in use: a mode in which the signal grows weaker over time and dies (volatile mode), and one in which the signal remains constant (non-volatile mode).

…

Once you get past the first two paragraphs in the Nanowerk news item, you find the ETH Zurich and Empa May 18, 2022 press releases by Fabio Begamin, in both cases, are identical (ETH is listed as the authoring agency on EurekAlert), (Note: A link has been removed in the following),

Just like in the brain

“These two operation modes are also found in the human brain,” John says. On the one hand, stimuli at the synapses are transmitted from neuron to neuron with biochemical neurotransmitters. These stimuli start out strong and then gradually become weaker. On the other hand, new synaptic connections to other neurons form in the brain while we learn. These connections are longer-​lasting.

John, who is a postdoc in the group headed by ETH Professor Maksym Kovalenko, was awarded an ETH fellowship for outstanding postdoctoral researchers in 2020. John conducted this research together with Yiğit Demirağ, a doctoral student in Professor Giacomo Indiveri’s group at the Institute for Neuroinformatics of the University of Zurich and ETH Zurich.

Semiconductor known from solar cells

The memristors the researchers have developed are made of halide perovskite nanocrystals, a semiconductor material known primarily from its use in photovoltaic cells. “The ‘nerve conduction’ in these new memristors is mediated by temporarily or permanently stringing together silver ions from an electrode to form a nanofilament penetrating the perovskite structure through which current can flow,” explains Kovalenko.

This process can be regulated to make the silver-​ion filament either thin, so that it gradually breaks back down into individual silver ions (volatile mode), or thick and permanent (non-​volatile mode). This is controlled by the intensity of the current conducted on the memristor: applying a weak current activates the volatile mode, while a strong current activates the non-​volatile mode.

New toolkit for neuroinformaticians

“To our knowledge, this is the first memristor that can be reliably switched between volatile and non-​volatile modes on demand,” Demirağ says. This means that in the future, computer chips can be manufactured with memristors that enable both modes. This is a significance advance because it is usually not possible to combine several different types of memristors on one chip.

Within the scope of the study, which they published in the journal Nature Communications, the researchers tested 25 of these new memristors and carried out 20,000 measurements with them. In this way, they were able to simulate a computational problem on a complex network. The problem involved classifying a number of different neuron spikes as one of four predefined patterns.

Before these memristors can be used in computer technology, they will need to undergo further optimisation.  However, such components are also important for research in neuroinformatics, as Indiveri points out: “These components come closer to real neurons than previous ones. As a result, they help researchers to better test hypotheses in neuroinformatics and hopefully gain a better understanding of the computing principles of real neuronal circuits in humans and animals.”

Here’s a link to and a citation for the paper,

Reconfigurable halide perovskite nanocrystal memristors for neuromorphic computing by Rohit Abraham John, Yiğit Demirağ, Yevhen Shynkarenko, Yuliia Berezovska, Natacha Ohannessian, Melika Payvand, Peng Zeng, Maryna I. Bodnarchuk, Frank Krumeich, Gökhan Kara, Ivan Shorubalko, Manu V. Nair, Graham A. Cooke, Thomas Lippert, Giacomo Indiveri & Maksym V. Kovalenko. Nature Communications volume 13, Article number: 2074 (2022) DOI: https://doi.org/10.1038/s41467-022-29727-1 Published: 19 April 2022

This paper is open access.

Connecting biological and artificial neurons (in UK, Switzerland, & Italy) over the web

Leave a reply
Caption: The virtual lab connecting Southampton, Zurich and Padova. Credit: University of Southampton

A February 26, 2020 University of Southampton press release (also on EurekAlert) describes this work,

Research on novel nanoelectronics devices led by the University of Southampton enabled brain neurons and artificial neurons to communicate with each other. This study has for the first time shown how three key emerging technologies can work together: brain-computer interfaces, artificial neural networks and advanced memory technologies (also known as memristors). The discovery opens the door to further significant developments in neural and artificial intelligence research.

Brain functions are made possible by circuits of spiking neurons, connected together by microscopic, but highly complex links called ‘synapses’. In this new study, published in the scientific journal Nature Scientific Reports, the scientists created a hybrid neural network where biological and artificial neurons in different parts of the world were able to communicate with each other over the internet through a hub of artificial synapses made using cutting-edge nanotechnology. This is the first time the three components have come together in a unified network.

During the study, researchers based at the University of Padova in Italy cultivated rat neurons in their laboratory, whilst partners from the University of Zurich and ETH Zurich created artificial neurons on Silicon microchips. The virtual laboratory was brought together via an elaborate setup controlling nanoelectronic synapses developed at the University of Southampton. These synaptic devices are known as memristors.

The Southampton based researchers captured spiking events being sent over the internet from the biological neurons in Italy and then distributed them to the memristive synapses. Responses were then sent onward to the artificial neurons in Zurich also in the form of spiking activity. The process simultaneously works in reverse too; from Zurich to Padova. Thus, artificial and biological neurons were able to communicate bidirectionally and in real time.

Themis Prodromakis, Professor of Nanotechnology and Director of the Centre for Electronics Frontiers at the University of Southampton said “One of the biggest challenges in conducting research of this kind and at this level has been integrating such distinct cutting edge technologies and specialist expertise that are not typically found under one roof. By creating a virtual lab we have been able to achieve this.”

The researchers now anticipate that their approach will ignite interest from a range of scientific disciplines and accelerate the pace of innovation and scientific advancement in the field of neural interfaces research. In particular, the ability to seamlessly connect disparate technologies across the globe is a step towards the democratisation of these technologies, removing a significant barrier to collaboration.

Professor Prodromakis added “We are very excited with this new development. On one side it sets the basis for a novel scenario that was never encountered during natural evolution, where biological and artificial neurons are linked together and communicate across global networks; laying the foundations for the Internet of Neuro-electronics. On the other hand, it brings new prospects to neuroprosthetic technologies, paving the way towards research into replacing dysfunctional parts of the brain with AI [artificial intelligence] chips.”

I’m fascinated by this work and after taking a look at the paper, I have to say, the paper is surprisingly accessible. In other words, I think I get the general picture. For example (from the Introduction to the paper; citation and link follow further down),

… To emulate plasticity, the memristor MR1 is operated as a two-terminal device through a control system that receives pre- and post-synaptic depolarisations from one silicon neuron (ANpre) and one biological neuron (BN), respectively. …

If I understand this properly, they’ve integrated a biological neuron and an artificial neuron in a single system across three countries.

For those who care to venture forth, here’s a link and a citation for the paper,

Memristive synapses connect brain and silicon spiking neurons by Alexantrou Serb, Andrea Corna, Richard George, Ali Khiat, Federico Rocchi, Marco Reato, Marta Maschietto, Christian Mayr, Giacomo Indiveri, Stefano Vassanelli & Themistoklis Prodromakis. Scientific Reports volume 10, Article number: 2590 (2020) DOI: https://doi.org/10.1038/s41598-020-58831-9 Published 25 February 2020

The paper is open access.