Tag Archives: Themistoklis Prodromakis

Neuromorphic engineering: an overview

Leave a reply

In a February 13, 2023 essay, Michael Berger who runs the Nanowerk website provides an overview of brainlike (neuromorphic) engineering.

This essay is the most extensive piece I’ve seen on Berger’s website and it covers everything from the reasons why scientists are so interested in mimicking the human brain to specifics about memristors. Here are a few excerpts (Note: Links have been removed),

Neuromorphic engineering is a cutting-edge field that focuses on developing computer hardware and software systems inspired by the structure, function, and behavior of the human brain. The ultimate goal is to create computing systems that are significantly more energy-efficient, scalable, and adaptive than conventional computer systems, capable of solving complex problems in a manner reminiscent of the brain’s approach.

This interdisciplinary field draws upon expertise from various domains, including neuroscience, computer science, electronics, nanotechnology, and materials science. Neuromorphic engineers strive to develop computer chips and systems incorporating artificial neurons and synapses, designed to process information in a parallel and distributed manner, akin to the brain’s functionality.

Key challenges in neuromorphic engineering encompass developing algorithms and hardware capable of performing intricate computations with minimal energy consumption, creating systems that can learn and adapt over time, and devising methods to control the behavior of artificial neurons and synapses in real-time.

Neuromorphic engineering has numerous applications in diverse areas such as robotics, computer vision, speech recognition, and artificial intelligence. The aspiration is that brain-like computing systems will give rise to machines better equipped to tackle complex and uncertain tasks, which currently remain beyond the reach of conventional computers.

It is essential to distinguish between neuromorphic engineering and neuromorphic computing, two related but distinct concepts. Neuromorphic computing represents a specific application of neuromorphic engineering, involving the utilization of hardware and software systems designed to process information in a manner akin to human brain function.

…

One of the major obstacles in creating brain-inspired computing systems is the vast complexity of the human brain. Unlike traditional computers, the brain operates as a nonlinear dynamic system that can handle massive amounts of data through various input channels, filter information, store key information in short- and long-term memory, learn by analyzing incoming and stored data, make decisions in a constantly changing environment, and do all of this while consuming very little power.

…

The Human Brain Project [emphasis mine], a large-scale research project launched in 2013, aims to create a comprehensive, detailed, and biologically realistic simulation of the human brain, known as the Virtual Brain. One of the goals of the project is to develop new brain-inspired computing technologies, such as neuromorphic computing.

…

The Human Brain Project has been funded by the European Union (1B Euros over 10 years starting in 2013 and sunsetting in 2023). From the Human Brain Project Media Invite,

The final Human Brain Project Summit 2023 will take place in Marseille, France, from March 28-31, 2023.

As the ten-year European Flagship Human Brain Project (HBP) approaches its conclusion in September 2023, the final HBP Summit will highlight the scientific achievements of the project at the interface of neuroscience and technology and the legacy that it will leave for the brain research community. …

One last excerpt from the essay,

Neuromorphic computing is a radical reimagining of computer architecture at the transistor level, modeled after the structure and function of biological neural networks in the brain. This computing paradigm aims to build electronic systems that attempt to emulate the distributed and parallel computation of the brain by combining processing and memory in the same physical location.

This is unlike traditional computing, which is based on von Neumann systems consisting of three different units: processing unit, I/O unit, and storage unit. This stored program architecture is a model for designing computers that uses a single memory to store both data and instructions, and a central processing unit to execute those instructions. This design, first proposed by mathematician and computer scientist John von Neumann, is widely used in modern computers and is considered to be the standard architecture for computer systems and relies on a clear distinction between memory and processing.

…

I found the diagram Berger Included with von Neumann’s design contrasted with a neuromorphic design illuminating,

A graphical comparison of the von Neumann and Neuromorphic architecture. Left: The von Neumann architecture used in traditional computers. The red lines depict the data communication bottleneck in the von Neumann architecture. Right: A graphical representation of a general neuromorphic architecture. In this architecture, the processing and memory is decentralized across different neuronal units(the yellow nodes) and synapses(the black lines connecting the nodes), creating a naturally parallel computing environment via the mesh-like structure. (Source: DOI: 10.1109/IS.2016.7737434) [downloaded from https://www.nanowerk.com/spotlight/spotid=62353.php]

Berger offers a very good overview and I recommend reading his February 13, 2023 essay on neuromorphic engineering with one proviso, Note: A link has been removed,

Many researchers in this field see memristors as a key device component for neuromorphic engineering. Memristor – or memory resistor – devices are non-volatile nanoelectronic memory devices that were first theorized [emphasis mine] by Leon Chua in the 1970’s. However, it was some thirty years later that the first practical device was fabricated in 2008 by a group led by Stanley Williams [sometimes cited as R. Stanley Williams] at HP Research Labs.

…

Chua wasn’t the first as he, himself, has noted. Chua arrived at his theory independently in the 1970s but Bernard Widrow theorized what he called a ‘memistor’ in the 1960s. In fact “Memristors: they are older than you think” is a May 22, 2012 posting which featured an article “Two centuries of memristors” by Themistoklis Prodromakis, Christofer Toumazou and Leon Chua published in Nature Materials.

Most of us try to get it right but we don’t always succeed. It’s always good practice to read everyone (including me) with a little skepticism.

Connecting biological and artificial neurons (in UK, Switzerland, & Italy) over the web

Leave a reply
Caption: The virtual lab connecting Southampton, Zurich and Padova. Credit: University of Southampton

A February 26, 2020 University of Southampton press release (also on EurekAlert) describes this work,

Research on novel nanoelectronics devices led by the University of Southampton enabled brain neurons and artificial neurons to communicate with each other. This study has for the first time shown how three key emerging technologies can work together: brain-computer interfaces, artificial neural networks and advanced memory technologies (also known as memristors). The discovery opens the door to further significant developments in neural and artificial intelligence research.

Brain functions are made possible by circuits of spiking neurons, connected together by microscopic, but highly complex links called ‘synapses’. In this new study, published in the scientific journal Nature Scientific Reports, the scientists created a hybrid neural network where biological and artificial neurons in different parts of the world were able to communicate with each other over the internet through a hub of artificial synapses made using cutting-edge nanotechnology. This is the first time the three components have come together in a unified network.

During the study, researchers based at the University of Padova in Italy cultivated rat neurons in their laboratory, whilst partners from the University of Zurich and ETH Zurich created artificial neurons on Silicon microchips. The virtual laboratory was brought together via an elaborate setup controlling nanoelectronic synapses developed at the University of Southampton. These synaptic devices are known as memristors.

The Southampton based researchers captured spiking events being sent over the internet from the biological neurons in Italy and then distributed them to the memristive synapses. Responses were then sent onward to the artificial neurons in Zurich also in the form of spiking activity. The process simultaneously works in reverse too; from Zurich to Padova. Thus, artificial and biological neurons were able to communicate bidirectionally and in real time.

Themis Prodromakis, Professor of Nanotechnology and Director of the Centre for Electronics Frontiers at the University of Southampton said “One of the biggest challenges in conducting research of this kind and at this level has been integrating such distinct cutting edge technologies and specialist expertise that are not typically found under one roof. By creating a virtual lab we have been able to achieve this.”

The researchers now anticipate that their approach will ignite interest from a range of scientific disciplines and accelerate the pace of innovation and scientific advancement in the field of neural interfaces research. In particular, the ability to seamlessly connect disparate technologies across the globe is a step towards the democratisation of these technologies, removing a significant barrier to collaboration.

Professor Prodromakis added “We are very excited with this new development. On one side it sets the basis for a novel scenario that was never encountered during natural evolution, where biological and artificial neurons are linked together and communicate across global networks; laying the foundations for the Internet of Neuro-electronics. On the other hand, it brings new prospects to neuroprosthetic technologies, paving the way towards research into replacing dysfunctional parts of the brain with AI [artificial intelligence] chips.”

I’m fascinated by this work and after taking a look at the paper, I have to say, the paper is surprisingly accessible. In other words, I think I get the general picture. For example (from the Introduction to the paper; citation and link follow further down),

… To emulate plasticity, the memristor MR1 is operated as a two-terminal device through a control system that receives pre- and post-synaptic depolarisations from one silicon neuron (ANpre) and one biological neuron (BN), respectively. …

If I understand this properly, they’ve integrated a biological neuron and an artificial neuron in a single system across three countries.

For those who care to venture forth, here’s a link and a citation for the paper,

Memristive synapses connect brain and silicon spiking neurons by Alexantrou Serb, Andrea Corna, Richard George, Ali Khiat, Federico Rocchi, Marco Reato, Marta Maschietto, Christian Mayr, Giacomo Indiveri, Stefano Vassanelli & Themistoklis Prodromakis. Scientific Reports volume 10, Article number: 2590 (2020) DOI: https://doi.org/10.1038/s41598-020-58831-9 Published 25 February 2020

The paper is open access.

The memristor as the ‘missing link’ in bioelectronic medicine?

Leave a reply

The last time I featured memrisors and a neuronal network it was in an April 22, 2016 posting about Russian research in that field. This latest work comes from the UK’s University of Southampton. From a Sept. 27, 2016 news item on phys.org,

New research, led by the University of Southampton, has demonstrated that a nanoscale device, called a memristor, could be the ‘missing link’ in the development of implants that use electrical signals from the brain to help treat medical conditions.

Monitoring neuronal cell activity is fundamental to neuroscience and the development of neuroprosthetics – biomedically engineered devices that are driven by neural activity. However, a persistent problem is the device being able to process the neural data in real-time, which imposes restrictive requirements on bandwidth, energy and computation capacity.

In a new study, published in Nature Communications, the researchers showed that memristors could provide real-time processing of neuronal signals (spiking events) leading to efficient data compression and the potential to develop more precise and affordable neuroprosthetics and bioelectronic medicines.

A Sept. 27, 2016 University of Southampton press release, which originated the news item, expands on the theme,

Memristors are electrical components that limit or regulate the flow of electrical current in a circuit and can remember the amount of charge that was flowing through it and retain the data, even when the power is turned off.

Lead author Isha Gupta, Postgraduate Research Student at the University of Southampton, said: “Our work can significantly contribute towards further enhancing the understanding of neuroscience, developing neuroprosthetics and bio-electronic medicines by building tools essential for interpreting the big data in a more effective way.”

The research team developed a nanoscale Memristive Integrating Sensor (MIS) into which they fed a series of voltage-time samples, which replicated neuronal electrical activity.

Acting like synapses in the brain, the metal-oxide MIS was able to encode and compress (up to 200 times) neuronal spiking activity recorded by multi-electrode arrays. Besides addressing the bandwidth constraints, this approach was also very power efficient – the power needed per recording channel was up to 100 times less when compared to current best practice.

Co-author Dr Themis Prodromakis, Reader in Nanoelectronics and EPSRC Fellow in Electronics and Computer Science at the University of Southampton said: “We are thrilled that we succeeded in demonstrating that these emerging nanoscale devices, despite being rather simple in architecture, possess ultra-rich dynamics that can be harnessed beyond the obvious memory applications to address the fundamental constraints in bandwidth and power that currently prohibit scaling neural interfaces beyond 1,000 recording channels.”

The Prodromakis Group at the University of Southampton is acknowledged as world-leading in this field, collaborating among others with Leon Chua (a Diamond Jubilee Visiting Academic at the University of Southampton), who theoretically predicted the existence of memristors in 1971.

Here’s a link to and a citation for the paper,

Real-time encoding and compression of neuronal spikes by metal-oxide memristors by Isha Gupta, Alexantrou Serb, Ali Khiat, Ralf Zeitler, Stefano Vassanelli, & Themistoklis Prodromakis. Nature Communications 7, Article number: 12805 doi:10.1038/ncomms12805 Published  26 September 2016

This is an open access paper.

For anyone who’s interested in better understanding memristors, there’s an interview with Forrest H Bennett III in my April 7, 2010 posting and you can always check Wikipedia.

Memristors: they are older than you think

2 Replies

I got an email this morning (May 22, 2012) informing me that an article, Two centuries of memristors by Themistoklis Prodromakis, Christofer Toumazou and Leon Chua, had just been published in the journal Nature Materials. The article situates memristors in an historical context stretching back to the 19th century. Sadly, the article is behind a paywall so I won’t be copying too much material but I will attempt to give you the flavour of the piece.

The focus is on 19th century scientists and their work with what we are now calling ‘memristors’.  Before moving on to the article, here’s a good definition of a memristor, from the Wikipedia essay (note: I have removed links and footnotes),

Memristor (…  a portmanteau of “memory resistor”) is a passive two-terminal electrical component envisioned as a fundamental non-linear circuit element relating charge and magnetic flux linkage. The memristor is currently under development by a team at Hewlett-Packard.

When current flows in one direction through the device, the electrical resistance increases; and when current flows in the opposite direction, the resistance decreases. When the current is stopped, the component retains the last resistance that it had, and when the flow of charge starts again, the resistance of the circuit will be what it was when it was last active. It has a regime of operation with an approximately linear charge-resistance relationship as long as the time-integral of the current stays within certain bounds.

This Wikipedia essay also offers an historical timeline, which starts in 1960 with Bernard Widrow and his memistor, adding very nicely to the discussion in the Nature Materials article which focuses on such 19th luminaries as Sir Michael Faraday, Hertha Ayrton, Alessandro Volta, and Humphry Davy, amongst others.  Here’s a helpful description of hysteresis and how it relates to the memristor from the article (note: I have removed footnotes),

The functional properties of memristors were first documented by Chua and later on by Chua and Kang, with their main fingerprint being a pinched-hysteresis loop when subjected to a bipolar periodic signal. This particular signature has been explicitly observed in a number of devices for more than one century, while it can be extrapolated for devices that appeared as early as the dawn of the nineteenth century.

Hysteresis is typically noticed in systems and devices that possess certain inertia, causing the value of a physical property to lag behind changes in the mechanism causing it, manifesting memory.

The authors go on to outline the various  scientists who have grappled with the ‘memristive effect’ dating back to two centuries ago.  They finish their essay with this (note:  I’ve removed footnotes),

The memristor is not an invention. Rather it is a description of a basic phenomenon of nature that manifests itself in various dissipative devices, made from different materials, internal structures and architectures. We end this historical narrative by noting that even though the memristor has seen its light of joy only recently in 2008, and has been recognized as the fourth circuit element along with the resistor, capacitor and inductor, it actually predates the resistor, which was formally published by Ohm in 1827, and the inductor, which was formally published by Faraday in 1831.

If you are at all interested in memristors and have access behind the paywall, I strongly recommend reading this paper not only for the historical context but for how the authors support their contention that the memristor is a fourth circuit element.

A contrasting perspective is offered by Blaise Mouttet (discussed in my Jan. 27, 2012 posting) who contends that the what we are now calling a ‘memristor’ is part of a larger class of variable resistance systems.