Tag Archives: Tomonobu Nakayama

Learning and remembering like a human brain: nanowire networks

It’s all about memory in this April 21, 2023 news item on Nanowerk, Note: A link has been removed,

An international team led by scientists at the University of Sydney has demonstrated nanowire networks can exhibit both short- and long-term memory like the human brain.

The research has been published today in the journal Science Advances (“Neuromorphic learning, working memory, and metaplasticity in nanowire networks”), led by Dr Alon Loeffler, who received his PhD in the School of Physics, with collaborators in Japan.

An April 24, 2023 University of Sydney (Australia) press release (also on EurekAlert but published April 21, 2023), which originated news item, offers more detail about the research,

“In this research we found higher-order cognitive function, which we normally associate with the human brain, can be emulated in non-biological hardware,” Dr Loeffler said.

“This work builds on our previous research in which we showed how nanotechnology could be used to build a brain-inspired electrical device with neural network-like circuitry and synapse-like signalling.

“Our current work paves the way towards replicating brain-like learning and memory in non-biological hardware systems and suggests that the underlying nature of brain-like intelligence may be physical.”

Nanowire networks are a type of nanotechnology typically made from tiny, highly conductive silver wires that are invisible to the naked eye, covered in a plastic material, which are scattered across each other like a mesh. The wires mimic aspects of the networked physical structure of a human brain.

Advances in nanowire networks could herald many real-world applications, such as improving robotics or sensor devices that need to make quick decisions in unpredictable environments.

“This nanowire network is like a synthetic neural network because the nanowires act like neurons, and the places where they connect with each other are analogous to synapses,” senior author Professor Zdenka Kuncic, from the School of Physics, said.

“Instead of implementing some kind of machine learning task, in this study Dr Loeffler has actually taken it one step further and tried to demonstrate that nanowire networks exhibit some kind of cognitive function.”

To test the capabilities of the nanowire network, the researchers gave it a test similar to a common memory task used in human psychology experiments, called the N-Back task.

For a person, the N-Back task might involve remembering a specific picture of a cat from a series of feline images presented in a sequence. An N-Back score of 7, the average for people, indicates the person can recognise the same image that appeared seven steps back.

When applied to the nanowire network, the researchers found it could ‘remember’ a desired endpoint in an electric circuit seven steps back, meaning a score of 7 in an N-Back test.

“What we did here is manipulate the voltages of the end electrodes to force the pathways to change, rather than letting the network just do its own thing. We forced the pathways to go where we wanted them to go,” Dr Loeffler said.

“When we implement that, its memory had much higher accuracy and didn’t really decrease over time, suggesting that we’ve found a way to strengthen the pathways to push them towards where we want them, and then the network remembers it.

“Neuroscientists think this is how the brain works, certain synaptic connections strengthen while others weaken, and that’s thought to be how we preferentially remember some things, how we learn and so on.”

The researchers said when the nanowire network is constantly reinforced, it reaches a point where that reinforcement is no longer needed because the information is consolidated into memory.

“It’s kind of like the difference between long-term memory and short-term memory in our brains,” Professor Kuncic said.

“If we want to remember something for a long period of time, we really need to keep training our brains to consolidate that, otherwise it just kind of fades away over time.

“One task showed that the nanowire network can store up to seven items in memory at substantially higher than chance levels without reinforcement training and near-perfect accuracy with reinforcement training.”

COI [Conflict of Interest] Statement

Professor Zdenka Kuncic is with Emergentia [can be found here], Inc. The authors declare that they have no other competing interests.

Caption: Neural network (left) nanowire network (right) Credit: Loeffler et al.

I have a link to and citation for the paper in Science Advances (another link and citation follows),

Neuromorphic learning, working memory, and metaplasticity in nanowire networks by Alon Loeffler, Adrian Diaz-Alvarez, Ruomin Zhu, Natesh Ganesh, James M. Shine, Tomonobu Nakayama, and Zdenka Kuncic. Science Advances 21 Apr 2023 Vol 9, Issue 16 DOI: 10.1126/sciadv.adg3289

This paper is open access.

Never having seen this organization’s (Zenodo.org) setup before I’m a little confused by it,

Neuromorphic Learning, Working Memory and Metaplasticity in Nanowire Networks by Loeffler, Alon; Diaz-Alvarez, Adrian; Zhu, Ruomin; Ganesh, Natesh; Shine, James. M; Nakayama, Tomonobu; Kuncic, Zdenka, https://zenodo.org/record/7633958#.ZEv_2EnMKpo Published: February 12, 2023

I’m not sure if they’re including an early version of the article (I don’t think so) but they do have other files, which are open access and they reference the Science Advances study published in April 2023.

It seems their focus is data, from the About Zenodo webpage,

Every last detail

To fully understand and reproduce research performed by others, it is necessary to have all the details. In the digital age, that means all the digital artefacts, which are all welcomed in Zenodo.

To be an effective catch­-all, that eliminates barriers to adopting data sharing practices, Zenodo does not impose any requirements on format, size, access restrictions or licence. Quite literally we wish there to be no reason for researchers not to share!

Data, software and other artefacts in support of publications may be the core, but equally welcome are the materials associated with the conferences, projects or the institutions themselves, all of which are necessary to understand the scholarly process.

Don’t wait until the publication date!

Publication may happen months or years after completion of the research, so collecting together all the research artefacts at that stage to publish openly is often challenging. Zenodo therefore offers the possibility to house closed and restricted content, so that artefacts can be captured and stored safely whilst the research is ongoing, such that nothing is missing when they are openly shared later in the research workflow.

Additionally, to help publishing, research materials for the review process can be safely uploaded to Zenodo in restricted records and then protected links can be shared with the reviewers. Content can also be embargoed and automatically opened when the associated paper is published.

To support all these use cases, the simple web interface is supplemented by a rich API which allows third ­party tools and services to use Zenodo as a backend in their workflow.

A tangle of silver nanowires for brain-like action

I’ve been meaning to get to this news item from late 2019 as it features work from a team that I’ve been following for a number of years now. First mentioned here in an October 17, 2011 posting, James Gimzewski has been working with researchers at the University of California at Los Angeles (UCLA) and researchers at Japan’s National Institute for Materials Science (NIMS) on neuromorphic computing.

This particular research had a protracted rollout with the paper being published in October 2019 and the last news item about it being published in mid-December 2019.

A December 17, 2029 news item on Nanowerk was the first to alert me to this new work (Note: A link has been removed),

UCLA scientists James Gimzewski and Adam Stieg are part of an international research team that has taken a significant stride toward the goal of creating thinking machines.

Led by researchers at Japan’s National Institute for Materials Science, the team created an experimental device that exhibited characteristics analogous to certain behaviors of the brain — learning, memorization, forgetting, wakefulness and sleep. The paper, published in Scientific Reports (“Emergent dynamics of neuromorphic nanowire networks”), describes a network in a state of continuous flux.

A December 16, 2019 UCLA news release, which originated the news item, offers more detail (Note: A link has been removed),

“This is a system between order and chaos, on the edge of chaos,” said Gimzewski, a UCLA distinguished professor of chemistry and biochemistry, a member of the California NanoSystems Institute at UCLA and a co-author of the study. “The way that the device constantly evolves and shifts mimics the human brain. It can come up with different types of behavior patterns that don’t repeat themselves.”

The research is one early step along a path that could eventually lead to computers that physically and functionally resemble the brain — machines that may be capable of solving problems that contemporary computers struggle with, and that may require much less power than today’s computers do.

The device the researchers studied is made of a tangle of silver nanowires — with an average diameter of just 360 nanometers. (A nanometer is one-billionth of a meter.) The nanowires were coated in an insulating polymer about 1 nanometer thick. Overall, the device itself measured about 10 square millimeters — so small that it would take 25 of them to cover a dime.

Allowed to randomly self-assemble on a silicon wafer, the nanowires formed highly interconnected structures that are remarkably similar to those that form the neocortex, the part of the brain involved with higher functions such as language, perception and cognition.

One trait that differentiates the nanowire network from conventional electronic circuits is that electrons flowing through them cause the physical configuration of the network to change. In the study, electrical current caused silver atoms to migrate from within the polymer coating and form connections where two nanowires overlap. The system had about 10 million of these junctions, which are analogous to the synapses where brain cells connect and communicate.

The researchers attached two electrodes to the brain-like mesh to profile how the network performed. They observed “emergent behavior,” meaning that the network displayed characteristics as a whole that could not be attributed to the individual parts that make it up. This is another trait that makes the network resemble the brain and sets it apart from conventional computers.

After current flowed through the network, the connections between nanowires persisted for as much as one minute in some cases, which resembled the process of learning and memorization in the brain. Other times, the connections shut down abruptly after the charge ended, mimicking the brain’s process of forgetting.

In other experiments, the research team found that with less power flowing in, the device exhibited behavior that corresponds to what neuroscientists see when they use functional MRI scanning to take images of the brain of a sleeping person. With more power, the nanowire network’s behavior corresponded to that of the wakeful brain.

The paper is the latest in a series of publications examining nanowire networks as a brain-inspired system, an area of research that Gimzewski helped pioneer along with Stieg, a UCLA research scientist and an associate director of CNSI.

“Our approach may be useful for generating new types of hardware that are both energy-efficient and capable of processing complex datasets that challenge the limits of modern computers,” said Stieg, a co-author of the study.

The borderline-chaotic activity of the nanowire network resembles not only signaling within the brain but also other natural systems such as weather patterns. That could mean that, with further development, future versions of the device could help model such complex systems.

In other experiments, Gimzewski and Stieg already have coaxed a silver nanowire device to successfully predict statistical trends in Los Angeles traffic patterns based on previous years’ traffic data.

Because of their similarities to the inner workings of the brain, future devices based on nanowire technology could also demonstrate energy efficiency like the brain’s own processing. The human brain operates on power roughly equivalent to what’s used by a 20-watt incandescent bulb. By contrast, computer servers where work-intensive tasks take place — from training for machine learning to executing internet searches — can use the equivalent of many households’ worth of energy, with the attendant carbon footprint.

“In our studies, we have a broader mission than just reprogramming existing computers,” Gimzewski said. “Our vision is a system that will eventually be able to handle tasks that are closer to the way the human being operates.”

The study’s first author, Adrian Diaz-Alvarez, is from the International Center for Material Nanoarchitectonics at Japan’s National Institute for Materials Science. Co-authors include Tomonobu Nakayama and Rintaro Higuchi, also of NIMS; and Zdenka Kuncic at the University of Sydney in Australia.

Caption: (a) Micrograph of the neuromorphic network fabricated by this research team. The network contains of numerous junctions between nanowires, which operate as synaptic elements. When voltage is applied to the network (between the green probes), current pathways (orange) are formed in the network. (b) A Human brain and one of its neuronal networks. The brain is known to have a complex network structure and to operate by means of electrical signal propagation across the network. Credit: NIMS

A November 11, 2019 National Institute for Materials Science (Japan) press release (also on EurekAlert but dated December 25, 2019) first announced the news,

An international joint research team led by NIMS succeeded in fabricating a neuromorphic network composed of numerous metallic nanowires. Using this network, the team was able to generate electrical characteristics similar to those associated with higher order brain functions unique to humans, such as memorization, learning, forgetting, becoming alert and returning to calm. The team then clarified the mechanisms that induced these electrical characteristics.

The development of artificial intelligence (AI) techniques has been rapidly advancing in recent years and has begun impacting our lives in various ways. Although AI processes information in a manner similar to the human brain, the mechanisms by which human brains operate are still largely unknown. Fundamental brain components, such as neurons and the junctions between them (synapses), have been studied in detail. However, many questions concerning the brain as a collective whole need to be answered. For example, we still do not fully understand how the brain performs such functions as memorization, learning and forgetting, and how the brain becomes alert and returns to calm. In addition, live brains are difficult to manipulate in experimental research. For these reasons, the brain remains a “mysterious organ.” A different approach to brain research?in which materials and systems capable of performing brain-like functions are created and their mechanisms are investigated?may be effective in identifying new applications of brain-like information processing and advancing brain science.

The joint research team recently built a complex brain-like network by integrating numerous silver (Ag) nanowires coated with a polymer (PVP) insulating layer approximately 1 nanometer in thickness. A junction between two nanowires forms a variable resistive element (i.e., a synaptic element) that behaves like a neuronal synapse. This nanowire network, which contains a large number of intricately interacting synaptic elements, forms a “neuromorphic network”. When a voltage was applied to the neuromorphic network, it appeared to “struggle” to find optimal current pathways (i.e., the most electrically efficient pathways). The research team measured the processes of current pathway formation, retention and deactivation while electric current was flowing through the network and found that these processes always fluctuate as they progress, similar to the human brain’s memorization, learning, and forgetting processes. The observed temporal fluctuations also resemble the processes by which the brain becomes alert or returns to calm. Brain-like functions simulated by the neuromorphic network were found to occur as the huge number of synaptic elements in the network collectively work to optimize current transport, in the other words, as a result of self-organized and emerging dynamic processes..

The research team is currently developing a brain-like memory device using the neuromorphic network material. The team intends to design the memory device to operate using fundamentally different principles than those used in current computers. For example, while computers are currently designed to spend as much time and electricity as necessary in pursuit of absolutely optimum solutions, the new memory device is intended to make a quick decision within particular limits even though the solution generated may not be absolutely optimum. The team also hopes that this research will facilitate understanding of the brain’s information processing mechanisms.

This project was carried out by an international joint research team led by Tomonobu Nakayama (Deputy Director, International Center for Materials Nanoarchitectonics (WPI-MANA), NIMS), Adrian Diaz Alvarez (Postdoctoral Researcher, WPI-MANA, NIMS), Zdenka Kuncic (Professor, School of Physics, University of Sydney, Australia) and James K. Gimzewski (Professor, California NanoSystems Institute, University of California Los Angeles, USA).

Here at last is a link to and a citation for the paper,

Emergent dynamics of neuromorphic nanowire networks by Adrian Diaz-Alvarez, Rintaro Higuchi, Paula Sanz-Leon, Ido Marcus, Yoshitaka Shingaya, Adam Z. Stieg, James K. Gimzewski, Zdenka Kuncic & Tomonobu Nakayama. Scientific Reports volume 9, Article number: 14920 (2019) DOI: https://doi.org/10.1038/s41598-019-51330-6 Published: 17 October 2019

This paper is open access.