Category Archives: electronics

Adaptive neural connectivity with an event-based architecture using photonic processors

Leave a reply

On first glance it looked like a set of matches. If there were more dimension, this could also have been a set pencils but no,

Caption: The chip contains almost 8,400 functioning artificial neurons from waveguide-coupled phase-change material. The researchers trained this neural network to distinguish between German and English texts on the basis of vowel frequency. Credit: Jonas Schütte / Pernice Group Courtesy: University of Münster

An October 23, 2023 news item on Nanowerk introduces research into a new approach to optical neural networks

A team of researchers headed by physicists Prof. Wolfram Pernice and Prof. Martin Salinga and computer specialist Prof. Benjamin Risse, all from the University of Münster, has developed a so-called event-based architecture, using photonic processors. In a similar way to the brain, this makes possible the continuous adaptation of the connections within the neural network.

Key Takeaways

Researchers have created a new computing architecture that mimics biological neural networks, using photonic processors for data transportation and processing.

The new system enables continuous adaptation of connections within the neural network, crucial for learning processes. This is known as both synaptic and structural plasticity.

Unlike traditional studies, the connections or synapses in this photonic neural network are not hardware-based but are coded based on optical pulse properties, allowing for a single chip to hold several thousand neurons.

Light-based processors in this system offer a much higher bandwidth and lower energy consumption compared to traditional electronic processors.

The researchers successfully tested the system using an evolutionary algorithm to differentiate between German and English texts based on vowel count, highlighting its potential for rapid and energy-efficient AI applications.

The Research

Modern computer models – for example for complex, potent AI applications – push traditional digital computer processes to their limits.

…

The person who edited the original press release, which is included in the news item in the above, is not credited.

Here’s the unedited original October 23, 2023 University of Münster press release (also on EurekAlert)

Modern computer models – for example for complex, potent AI applications – push traditional digital computer processes to their limits. New types of computing architecture, which emulate the working principles of biological neural networks, hold the promise of faster, more energy-efficient data processing. A team of researchers has now developed a so-called event-based architecture, using photonic processors with which data are transported and processed by means of light. In a similar way to the brain, this makes possible the continuous adaptation of the connections within the neural network. This changeable connections are the basis for learning processes. For the purposes of the study, a team working at Collaborative Research Centre 1459 (“Intelligent Matter”) – headed by physicists Prof. Wolfram Pernice and Prof. Martin Salinga and computer specialist Prof. Benjamin Risse, all from the University of Münster – joined forces with researchers from the Universities of Exeter and Oxford in the UK. The study has been published in the journal “Science Advances”.

What is needed for a neural network in machine learning are artificial neurons which are activated by external excitatory signals, and which have connections to other neurons. The connections between these artificial neurons are called synapses – just like the biological original. For their study, the team of researchers in Münster used a network consisting of almost 8,400 optical neurons made of waveguide-coupled phase-change material, and the team showed that the connection between two each of these neurons can indeed become stronger or weaker (synaptic plasticity), and that new connections can be formed, or existing ones eliminated (structural plasticity). In contrast to other similar studies, the synapses were not hardware elements but were coded as a result of the properties of the optical pulses – in other words, as a result of the respective wavelength and of the intensity of the optical pulse. This made it possible to integrate several thousand neurons on one single chip and connect them optically.

In comparison with traditional electronic processors, light-based processors offer a significantly higher bandwidth, making it possible to carry out complex computing tasks, and with lower energy consumption. This new approach consists of basic research. “Our aim is to develop an optical computing architecture which in the long term will make it possible to compute AI applications in a rapid and energy-efficient way,” says Frank Brückerhoff-Plückelmann, one of the lead authors.

Methodology: The non-volatile phase-change material can be switched between an amorphous structure and a crystalline structure with a highly ordered atomic lattice. This feature allows permanent data storage even without an energy supply. The researchers tested the performance of the neural network by using an evolutionary algorithm to train it to distinguish between German and English texts. The recognition parameter they used was the number of vowels in the text.

The researchers received financial support from the German Research Association, the European Commission and “UK Research and Innovation”.

Here’s a link to and a citation for the paper,

Event-driven adaptive optical neural network by Frank Brückerhoff-Plückelmann, Ivonne Bente, Marlon Becker, Niklas Vollmar, Nikolaos Farmakidis, Emma Lomonte, Francesco Lenzini, C. David Wright, Harish Bhaskaran, Martin Salinga, Benjamin Risse, and Wolfram H. P. Pernice. Science Advances 20 Oct 2023 Vol 9, Issue 42 DOI: 10.1126/sciadv.adi9127

This paper is open access.

Living technology possibilities

Leave a reply

Before launching into the possibilities, here are two descriptions of ‘living technology’ from the European Centre for Living Technology’s (ECLT) homepage,

Goals

Promote, carry out and coordinate research activities and the diffusion of scientific results in the field of living technology. The scientific areas for living technology are the nano-bio-technologies, self-organizing and evolving information and production technologies, and adaptive complex systems.

…

History

Founded in 2004 the European Centre for Living Technology is an international and interdisciplinary research centre established as an inter-university consortium, currently involving 18 European and extra-European institutional affiliates.

The Centre is devoted to the study of technologies that exhibit life-like properties including self-organization, adaptability and the capacity to evolve.

…

Despite the reference to “nano-bio-technologies,” this October 11, 2023 news item on ScienceDaily focuses on microscale living technology,

It is noIn a recent article in the high-profile journal “Advanced Materials,” researchers in Chemnitz show just how close and necessary the transition to sustainable living technology is, based on the morphogenesis of self-assembling microelectronic modules, strengthening the recent membership of Chemnitz University of Technology with the European Centre for Living Technology (ECLT) in Venice.

…

An October 11, 2023 Chemnitz University of Technology (Technische Universität Chemnitz; TU Chemnitz) press release (also on EurekAlert), which originated the news item, delves further into the topic, Note: Links have been removed,

It is now apparent that the mass-produced artefacts of technology in our increasingly densely populated world – whether electronic devices, cars, batteries, phones, household appliances, or industrial robots – are increasingly at odds with the sustainable bounded ecosystems achieved by living organisms based on cells over millions of years. Cells provide organisms with soft and sustainable environmental interactions with complete recycling of material components, except in a few notable cases like the creation of oxygen in the atmosphere, and of the fossil fuel reserves of oil and coal (as a result of missing biocatalysts). However, the fantastic information content of biological cells (gigabits of information in DNA alone) and the complexities of protein biochemistry for metabolism seem to place a cellular approach well beyond the current capabilities of technology, and prevent the development of intrinsically sustainable technology.

SMARTLETs: tiny shape-changing modules that collectively self-organize to larger more complex systems

A recent perspective review published in the very high impact journal Advanced Materials this month [October 2023] by researchers at the Research Center for Materials, Architectures and Integration of Nanomembranes (MAIN) of Chemnitz University of Technology, shows how a novel form of high-information-content Living Technology is now within reach, based on microrobotic electronic modules called SMARTLETs, which will soon be capable of self-assembling into complex artificial organisms. The research belongs to the new field of Microelectronic Morphogenesis, the creation of form under microelectronic control, and builds on work over the previous years at Chemnitz University of Technology to construct self-folding and self-locomoting thin film electronic modules, now carrying tiny silicon chiplets between the folds, for a massive increase in information processing capabilities. Sufficient information can now be stored in each module to encode not only complex functions but fabrication recipes (electronic genomes) for clean rooms to allow the modules to be copied and evolved like cells, but safely because of the gating of reproduction through human operated clean room facilities.

Electrical self-awareness during self-assembly

In addition, the chiplets can provide neuromorphic learning capabilities allowing them to improve performance during operation. A further key feature of the specific self-assembly of these modules, based on matching physical bar codes, is that electrical and fluidic connections can be achieved between modules. These can then be employed, to make the electronic chiplets on board “aware” of the state of assembly, and of potential errors, allowing them to direct repair, correct mis-assembly, induce disassembly and form collective functions spanning many modules. Such functions include extended communication (antennae), power harvesting and redistribution, remote sensing, material redistribution etc.

So why is this technology vital for sustainability?

The complete digital fab description for modules, for which actually only a limited number of types are required even for complex organisms, allows their material content, responsible originator and environmentally relevant exposure all to be read out. Prof. Dagmar Nuissl-Gesmann from the Law Department at Chemnitz University of Technology observes that “this fine-grained documentation of responsibility intrinsic down to microscopic scales will be a game changer in allowing legal assignment of environmental and social responsibility for our technical artefacts”.

Furthermore, the self-locomotion and self-assembly-disassembly capabilities allows the modules to self-sort for recycling. Modules can be regained, reused, reconfigured, and redeployed in different artificial organisms. If they are damaged, then their limited and documented types facilitate efficient custom recycling of materials with established and optimized protocols for these sorted and now identical entities. These capabilities complement the other more obvious advantages in terms of design development and reuse in this novel reconfigurable media. As Prof. Marlen Arnold, an expert in Sustainability of the Faculty of Economics and Business Administration observes, “Even at high volumes of deployment use, these properties could provide this technology with a hitherto unprecedented level of sustainability which would set the bar for future technologies to share our planet safely with us.”

Contribution to European Living Technology

This research is a first contribution of MAIN/Chemnitz University of Technology, as a new member of the European Centre for Living Technology ECLT, based in Venice,” says Prof. Oliver G. Schmidt, Scientific Director of the Research Center MAIN and adds that “It’s fantastic to see that our deep collaboration with ECLT is paying off so quickly with immediate transdisciplinary benefit for several scientific communities.” “Theoretical research at the ECLT has been urgently in need of novel technology systems able to implement the core properties of living systems.” comments Prof. John McCaskill, coauthor of the paper, and a grounding director of the ECLT in 2004.

Here’s a link to and a citation for the researchers’ perspective paper,

Microelectronic Morphogenesis: Smart Materials with Electronics Assembling into Artificial Organisms by John S. McCaskill, Daniil Karnaushenko, Minshen Zhu, Oliver G. Schmidt. Advanced Materials DOI: https://doi.org/10.1002/adma.202306344 First published: 09 October 2023

This paper is open access.

A formal theory for neuromorphic (brainlike) computing hardware needed

Leave a reply

This is one my older pieces as the information dates back to October 2023 but neuromorphic computing is one of my key interests and I’m particularly interested to see the upsurge in the discussion of hardware, here goes. From an October 17, 2023 news item on Nanowerk,

There is an intense, worldwide search for novel materials to build computer microchips with that are not based on classic transistors but on much more energy-saving, brain-like components. However, whereas the theoretical basis for classic transistor-based digital computers is solid, there are no real theoretical guidelines for the creation of brain-like computers.

Such a theory would be absolutely necessary to put the efforts that go into engineering new kinds of microchips on solid ground, argues Herbert Jaeger, Professor of Computing in Cognitive Materials at the University of Groningen [Netherlands].

Key Takeaways
Scientists worldwide are searching for new materials to build energy-saving, brain-like computer microchips as classic transistor miniaturization reaches its physical limit.

Theoretical guidelines for brain-like computers are lacking, making it crucial for advancements in the field.

The brain’s versatility and robustness serve as an inspiration, despite limited knowledge about its exact workings.

A recent paper suggests that a theory for non-digital computers should focus on continuous, analogue signals and consider the characteristics of new materials.

Bridging gaps between diverse scientific fields is vital for developing a foundational theory for neuromorphic computing..

…

An October 17, 2023 University of Groningen press release (also on EurekAlert), which originated the news item, provides more context for this proposal,

Computers have, so far, relied on stable switches that can be off or on, usually transistors. These digital computers are logical machines and their programming is also based on logical reasoning. For decades, computers have become more powerful by further miniaturization of the transistors, but this process is now approaching a physical limit. That is why scientists are working to find new materials to make more versatile switches, which could use more values than just the digitals 0 or 1.

Dangerous pitfall

Jaeger is part of the Groningen Cognitive Systems and Materials Center (CogniGron), which aims to develop neuromorphic (i.e. brain-like) computers. CogniGron is bringing together scientists who have very different approaches: experimental materials scientists and theoretical modelers from fields as diverse as mathematics, computer science, and AI. Working closely with materials scientists has given Jaeger a good idea of the challenges that they face when trying to come up with new computational materials, while it has also made him aware of a dangerous pitfall: there is no established theory for the use of non-digital physical effects in computing systems.

Our brain is not a logical system. We can reason logically, but that is only a small part of what our brain does. Most of the time, it must work out how to bring a hand to a teacup or wave to a colleague on passing them in a corridor. ‘A lot of the information-processing that our brain does is this non-logical stuff, which is continuous and dynamic. It is difficult to formalize this in a digital computer,’ explains Jaeger. Furthermore, our brains keep working despite fluctuations in blood pressure, external temperature, or hormone balance, and so on. How is it possible to create a computer that is as versatile and robust? Jaeger is optimistic: ‘The simple answer is: the brain is proof of principle that it can be done.’

Neurons

The brain is, therefore, an inspiration for materials scientists. Jaeger: ‘They might produce something that is made from a few hundred atoms and that will oscillate, or something that will show bursts of activity. And they will say: “That looks like how neurons work, so let’s build a neural network”.’ But they are missing a vital bit of knowledge here. ‘Even neuroscientists don’t know exactly how the brain works. This is where the lack of a theory for neuromorphic computers is problematic. Yet, the field doesn’t appear to see this.’

In a paper published in Nature Communications on 16 August, Jaeger and his colleagues Beatriz Noheda (scientific director of CogniGron) and Wilfred G. van der Wiel (University of Twente) present a sketch of what a theory for non-digital computers might look like. They propose that instead of stable 0/1 switches, the theory should work with continuous, analogue signals. It should also accommodate the wealth of non-standard nanoscale physical effects that the materials scientists are investigating.

Sub-theories

Something else that Jaeger has learned from listening to materials scientists is that devices from these new materials are difficult to construct. Jaeger: ‘If you make a hundred of them, they will not all be identical.’ This is actually very brain-like, as our neurons are not all exactly identical either. Another possible issue is that the devices are often brittle and temperature-sensitive, continues Jaeger. ‘Any theory for neuromorphic computing should take such characteristics into account.’

Importantly, a theory underpinning neuromorphic computing will not be a single theory but will be constructed from many sub-theories (see image below). Jaeger: ‘This is in fact how digital computer theory works as well, it is a layered system of connected sub-theories.’ Creating such a theoretical description of neuromorphic computers will require close collaboration of experimental materials scientists and formal theoretical modellers. Jaeger: ‘Computer scientists must be aware of the physics of all these new materials [emphasis mine] and materials scientists should be aware of the fundamental concepts in computing.’

Blind spots

Bridging this divide between materials science, neuroscience, computing science, and engineering is exactly why CogniGron was founded at the University of Groningen: it brings these different groups together. ‘We all have our blind spots,’ concludes Jaeger. ‘And the biggest gap in our knowledge is a foundational theory for neuromorphic computing. Our paper is a first attempt at pointing out how such a theory could be constructed and how we can create a common language.’

Here’s a link to and a citation for the paper,

Toward a formal theory for computing machines made out of whatever physics offers by Herbert Jaeger, Beatriz Noheda & Wilfred G. van der Wiel. Nature Communications volume 14, Article number: 4911 (2023) DOI: https://doi.org/10.1038/s41467-023-40533-1 Published: 16 August 2023

This paper is open access and there’s a 76 pp. version, “Toward a formal theory for computing machines made out of whatever physics offers: extended version” (emphasis mine) available on arXchiv.

Caption: A general theory of physical computing systems would comprise existing theories as special cases. Figure taken from an extended version of the Nature Comm paper on arXiv. Credit: Jaeger et al. / University of Groningen

With regard to new materials for neuromorphic computing, my January 4, 2024 posting highlights a proposed quantum material for this purpose.

A hardware (neuromorphic and quantum) proposal for handling increased AI workload

Leave a reply

It’s been a while since I’ve featured anything from Purdue University (Indiana, US). From a November 7, 2023 news item on Nanowerk, Note Links have been removed,

Technology is edging closer and closer to the super-speed world of computing with artificial intelligence. But is the world equipped with the proper hardware to be able to handle the workload of new AI technological breakthroughs?

Key Takeaways
Current AI technologies are strained by the limitations of silicon-based computing hardware, necessitating new solutions.

Research led by Erica Carlson [Purdue University] suggests that neuromorphic [brainlike] architectures, which replicate the brain’s neurons and synapses, could revolutionize computing efficiency and power.

Vanadium oxides have been identified as a promising material for creating artificial neurons and synapses, crucial for neuromorphic computing.

Innovative non-volatile memory, observed in vanadium oxides, could be the key to more energy-efficient and capable AI hardware.

Future research will explore how to optimize the synaptic behavior of neuromorphic materials by controlling their memory properties.

…

The colored landscape above shows a transition temperature map of VO2 (pink surface) as measured by optical microscopy. This reveals the unique way that this neuromorphic quantum material [emphasis mine] stores memory like a synapse. Image credit: Erica Carlson, Alexandre Zimmers, and Adobe Stock

An October 13, 2023 Purdue University news release (also on EurekAlert but published November 6, 2023) by Cheryl Pierce, which originated the news item, provides more detail about the work, Note: A link has been removed,

“The brain-inspired codes of the AI revolution are largely being run on conventional silicon computer architectures which were not designed for it,” explains Erica Carlson, 150th Anniversary Professor of Physics and Astronomy at Purdue University.

A joint effort between Physicists from Purdue University, University of California San Diego (USCD) and École Supérieure de Physique et de Chimie Industrielles (ESPCI) in Paris, France, believe they may have discovered a way to rework the hardware…. [sic] By mimicking the synapses of the human brain.  They published their findings, “Spatially Distributed Ramp Reversal Memory in VO2” in Advanced Electronic Materials which is featured on the back cover of the October 2023 edition.

New paradigms in hardware will be necessary to handle the complexity of tomorrow’s computational advances. According to Carlson, lead theoretical scientist of this research, “neuromorphic architectures hold promise for lower energy consumption processors, enhanced computation, fundamentally different computational modes, native learning and enhanced pattern recognition.”

Neuromorphic architecture basically boils down to computer chips mimicking brain behavior.  Neurons are cells in the brain that transmit information. Neurons have small gaps at their ends that allow signals to pass from one neuron to the next which are called synapses. In biological brains, these synapses encode memory. This team of scientists concludes that vanadium oxides show tremendous promise for neuromorphic computing because they can be used to make both artificial neurons and synapses.

“The dissonance between hardware and software is the origin of the enormously high energy cost of training, for example, large language models like ChatGPT,” explains Carlson. “By contrast, neuromorphic architectures hold promise for lower energy consumption by mimicking the basic components of a brain: neurons and synapses. Whereas silicon is good at memory storage, the material does not easily lend itself to neuron-like behavior. Ultimately, to provide efficient, feasible neuromorphic hardware solutions requires research into materials with radically different behavior from silicon – ones that can naturally mimic synapses and neurons. Unfortunately, the competing design needs of artificial synapses and neurons mean that most materials that make good synaptors fail as neuristors, and vice versa. Only a handful of materials, most of them quantum materials, have the demonstrated ability to do both.”

The team relied on a recently discovered type of non-volatile memory which is driven by repeated partial temperature cycling through the insulator-to-metal transition. This memory was discovered in vanadium oxides.

Alexandre Zimmers, lead experimental scientist from Sorbonne University and École Supérieure de Physique et de Chimie Industrielles, Paris, explains, “Only a few quantum materials are good candidates for future neuromorphic devices, i.e., mimicking artificial synapses and neurons. For the first time, in one of them, vanadium dioxide, we can see optically what is changing in the material as it operates as an artificial synapse. We find that memory accumulates throughout the entirety of the sample, opening new opportunities on how and where to control this property.”

“The microscopic videos show that, surprisingly, the repeated advance and retreat of metal and insulator domains causes memory to be accumulated throughout the entirety of the sample, rather than only at the boundaries of domains,” explains Carlson. “The memory appears as shifts in the local temperature at which the material transitions from insulator to metal upon heating, or from metal to insulator upon cooling. We propose that these changes in the local transition temperature accumulate due to the preferential diffusion of point defects into the metallic domains that are interwoven through the insulator as the material is cycled partway through the transition.”

Now that the team has established that vanadium oxides are possible candidates for future neuromorphic devices, they plan to move forward in the next phase of their research.

“Now that we have established a way to see inside this neuromorphic material, we can locally tweak and observe the effects of, for example, ion bombardment on the material’s surface,” explains Zimmers. “This could allow us to guide the electrical current through specific regions in the sample where the memory effect is at its maximum. This has the potential to significantly enhance the synaptic behavior of this neuromorphic material.”

There’s a very interesting 16 mins. 52 secs. video embedded in the October 13, 2023 Purdue University news release. In an interview with Dr. Erica Carlson who hosts The Quantum Age website and video interviews on its YouTube Channel, Alexandre Zimmers takes you from an amusing phenomenon observed by 19th century scientists through the 20th century where it becomes of more interest as the nanscale phenonenon can be exploited (sonar, scanning tunneling microscopes, singing birthday cards, etc.) to the 21st century where we are integrating this new information into a quantum* material for neuromorphic hardware.

Here’s a link to and a citation for the paper,

Spatially Distributed Ramp Reversal Memory in VO2 by Sayan Basak, Yuxin Sun, Melissa Alzate Banguero, Pavel Salev, Ivan K. Schuller, Lionel Aigouy, Erica W. Carlson, Alexandre Zimmers. Advanced Electronic Materials Volume 9, Issue 10 October 2023 2300085 DOI: https://doi.org/10.1002/aelm.202300085 First published: 10 July 2023

This paper is open access.

There’s a lot of research into neuromorphic hardware, here’s a sampling of some of my most recent posts on the topic,

There’s more, just use ‘neuromorphic hardware’ for your search term.

*’meta’ changed to ‘quantum’ on January 8, 2024.

Powered with salt water

Leave a reply

Apparently, salt water can be used both in the production of fusion energy (a form of nuclear energy) and, according to new research from the University of Illinois into a nanofluidic device, electricity. From a September 22, 2023 University of Illinois news release (also on EurekAlert),

There is a largely untapped energy source along the world’s coastlines: the difference in salinity between seawater and freshwater. A new nanodevice can harness this difference to generate power.

A team of researchers at the University of Illinois Urbana-Champaign has reported a design for a nanofluidic device capable of converting ionic flow into usable electric power in the journal Nano Energy. The team believes that their device could be used to extract power from the natural ionic flows at seawater-freshwater boundaries.

“While our design is still a concept at this stage, it is quite versatile and already shows strong potential for energy applications,” said Jean-Pierre Leburton, a U. of I. professor of electrical & computer engineering and the project lead. “It began with an academic question – ‘Can a nanoscale solid-state device extract energy from ionic flow?’ – but our design exceeded our expectations and surprised us in many ways.”

When two bodies of water with different salinity meet, such as where a river empties into an ocean, salt molecules naturally flow from higher concentration to lower concentration. The energy of these flows can be harvested because they consist of electrically charged particles called ions that form from the dissolved salt.

Leburton’s group designed a nanoscale semiconductor device that takes advantage of a phenomenon called “Coulomb drag” between flowing ions and electric charges in the device. When the ions flow through a narrow channel in the device, electric forces cause the device charges to move from one side to the other creating voltage and electric current.

The researchers found two surprising behaviors when they simulated their device. First, while they expected that Coulomb drag would primarily occur through the attractive force between opposite electric charges, the simulations indicated that the device works equally well if the electric forces are repulsive. Both positively and negatively charged ions contribute to drag.

“Just as noteworthy, our study indicates that there is an amplification effect” said Mingye Xiong, a graduate student in Leburton’s group and the study’s lead author. “Since the moving ions are so massive compared to the device charges, the ions impart large amounts of momentum to the charges, amplifying the underlying current.”

The researchers also found that these effects are independent of the specific channel configuration as well as the choice of materials, provided the channel diameter is narrow enough to ensure proximity between the ions and the charges.

The researchers are in the process of patenting their findings, and they are studying how arrays of these devices could scale for practical power generation.

“We believe that the power density of a device array could meet or exceed that of solar cells,” Leburton said. “And that’s not to mention the potential applications in other fields like biomedical sensing and nanofluidics.”

Here’s a link to and a citation for the paper,

Ionic coulomb drag in nanofluidic semiconductor channels for energy harvest by Mingye Xiong, Kewei Song, Jean-Pierre Leburton. Nano Energy Volume 117, 1 December 2023, 108860 DOI: https://doi.org/10.1016/j.nanoen.2023.108860

This paper is behind a paywall.

Shape-changing speaker (aka acoustic swarms) for sound control

Leave a reply

To alleviate any concerns, these swarms are not kin to Michael Crichton’s swarms in his 2002 novel, Prey or his 2011 novel, Micro (published after his death).

A September 21, 2023 news item on ScienceDaily announces this ‘acoustic swarm’ research,

In virtual meetings, it’s easy to keep people from talking over each other. Someone just hits mute. But for the most part, this ability doesn’t translate easily to recording in-person gatherings. In a bustling cafe, there are no buttons to silence the table beside you.

The ability to locate and control sound — isolating one person talking from a specific location in a crowded room, for instance — has challenged researchers, especially without visual cues from cameras.

A team led by researchers at the University of Washington has developed a shape-changing smart speaker, which uses self-deploying microphones to divide rooms into speech zones and track the positions of individual speakers. With the help of the team’s deep-learning algorithms, the system lets users mute certain areas or separate simultaneous conversations, even if two adjacent people have similar voices. Like a fleet of Roombas, each about an inch in diameter, the microphones automatically deploy from, and then return to, a charging station. This allows the system to be moved between environments and set up automatically. In a conference room meeting, for instance, such a system might be deployed instead of a central microphone, allowing better control of in-room audio.

The team published its findings Sept. 21 [2023] in Nature Communications.

…

A September 21, 2023 University of Washington (state) news release (also on EurekAlert), which originated the news item, delves further into the work, Note: Links have been removed,

“If I close my eyes and there are 10 people talking in a room, I have no idea who’s saying what and where they are in the room exactly. That’s extremely hard for the human brain to process. Until now, it’s also been difficult for technology,” said co-lead author Malek Itani, a UW doctoral student in the Paul G. Allen School of Computer Science & Engineering. “For the first time, using what we’re calling a robotic ‘acoustic swarm,’ we’re able to track the positions of multiple people talking in a room and separate their speech.”

Previous research on robot swarms has required using overhead or on-device cameras, projectors or special surfaces. The UW team’s system is the first to accurately distribute a robot swarm using only sound.

The team’s prototype consists of seven small robots that spread themselves across tables of various sizes. As they move from their charger, each robot emits a high frequency sound, like a bat navigating, using this frequency and other sensors to avoid obstacles and move around without falling off the table. The automatic deployment allows the robots to place themselves for maximum accuracy, permitting greater sound control than if a person set them. The robots disperse as far from each other as possible since greater distances make differentiating and locating people speaking easier. Today’s consumer smart speakers have multiple microphones, but clustered on the same device, they’re too close to allow for this system’s mute and active zones.

“If I have one microphone a foot away from me, and another microphone two feet away, my voice will arrive at the microphone that’s a foot away first. If someone else is closer to the microphone that’s two feet away, their voice will arrive there first,” said co-lead author Tuochao Chen, a UW doctoral student in the Allen School. “We developed neural networks that use these time-delayed signals to separate what each person is saying and track their positions in a space. So you can have four people having two conversations and isolate any of the four voices and locate each of the voices in a room.”

The team tested the robots in offices, living rooms and kitchens with groups of three to five people speaking. Across all these environments, the system could discern different voices within 1.6 feet (50 centimeters) of each other 90% of the time, without prior information about the number of speakers. The system was able to process three seconds of audio in 1.82 seconds on average — fast enough for live streaming, though a bit too long for real-time communications such as video calls.

As the technology progresses, researchers say, acoustic swarms might be deployed in smart homes to better differentiate people talking with smart speakers. That could potentially allow only people sitting on a couch, in an “active zone,” to vocally control a TV, for example.

Researchers plan to eventually make microphone robots that can move around rooms, instead of being limited to tables. The team is also investigating whether the speakers can emit sounds that allow for real-world mute and active zones, so people in different parts of a room can hear different audio. The current study is another step toward science fiction technologies, such as the “cone of silence” in “Get Smart” and“Dune,” the authors write.

Of course, any technology that evokes comparison to fictional spy tools will raise questions of privacy. Researchers acknowledge the potential for misuse, so they have included guards against this: The microphones navigate with sound, not an onboard camera like other similar systems. The robots are easily visible and their lights blink when they’re active. Instead of processing the audio in the cloud, as most smart speakers do, the acoustic swarms process all the audio locally, as a privacy constraint. And even though some people’s first thoughts may be about surveillance, the system can be used for the opposite, the team says.

“It has the potential to actually benefit privacy, beyond what current smart speakers allow,” Itani said. “I can say, ‘Don’t record anything around my desk,’ and our system will create a bubble 3 feet around me. Nothing in this bubble would be recorded. Or if two groups are speaking beside each other and one group is having a private conversation, while the other group is recording, one conversation can be in a mute zone, and it will remain private.”

Takuya Yoshioka, a principal research manager at Microsoft, is a co-author on this paper, and Shyam Gollakota, a professor in the Allen School, is a senior author. The research was funded by a Moore Inventor Fellow award.

Two of the paper`s authors, Malek Itani and Tuochao Chen, have written a ‘Behind the Paper’ article for Nature.com’s Electrical and Electronic Engineering Community, from their September 21, 2023 posting,

Sound is a versatile medium. In addition to being one of the primary means of communication for us humans, it serves numerous purposes for organisms across the animal kingdom. Particularly, many animals use sound to localize themselves and navigate in their environment. Bats, for example, emit ultrasonic sound pulses to move around and find food in the dark. Similar behavior can be observed in Beluga whales to avoid obstacles and locate one other.

Various animals also have a tendency to cluster together into swarms, forming a unit greater than the sum of its parts. Famously, bees agglomerate into swarms to more efficiently search for a new colony. Birds flock to evade predators. These behaviors have caught the attention of scientists for quite some time, inspiring a handful of models for crowd control, optimization and even robotics. 

A key challenge in building robot swarms for practical purposes is the ability for the robots to localize themselves, not just within the swarm, but also relative to other important landmarks. …

Here’s a link to and a citation for the paper,

Creating speech zones with self-distributing acoustic swarms by Malek Itani, Tuochao Chen, Takuya Yoshioka & Shyamnath Gollakota. Nature Communications volume 14, Article number: 5684 (2023) DOI: https://doi.org/10.1038/s41467-023-40869-8 Published: 21 September 2023

This paper is open access.

(nano) Rust and magnets from the Canadian Light Source

Leave a reply

An October 5, 2023 news item on phys.org highlights research from the Canadian Light Source (CLS, also known as, the synchrotron located in Saskatoon, Saskatchewan), Note: A link has been removed,

Every motor we use needs a magnet. University of Manitoba researcher Rachel Nickel is studying how rust could make those magnets cheaper and easier to produce.

Her most recent paper, published in the journal Nano Letters, explores a unique type of iron oxide nanoparticle. This material has special magnetic and electric features that could make it useful. It even has potential as a permanent magnet, which we use in car and airplane motors.

What sets it apart from other magnets is that it’s made from two of the most common elements found on earth: iron and oxygen. Right now, we use magnets made out of some of the rarest elements on the planet.

…

An October 5, 2023 CLS news release (also received via email) by Victoria Martinez, which originated the news item, provides more detail,

“The ability to produce magnets without rare earth elements [emphasis mine] is incredibly exciting,” says Nickel. “Almost everything that we use that has a motor where we need to start a motion relies on a permanent magnet”.

Researchers only started to understand this unique type of rust, called epsilon iron oxide, in the last 20 years.

“Now, what’s special about epsilon iron oxide is it only exists in the nanoscale,” says Nickel. “It’s basically fancy dust. But it is fancy dust with such incredible potential.”

In order to use it in everyday technology, researchers like Nickel need to understand its structure. To study epsilon iron oxide’s structure in different sizes, Nickel and colleagues collected data at the Advanced Photon Source (APS) in Illinois, thanks to the facility’s partnership with the Canadian Light Source (CLS) at the University of Saskatchewan. As the particle sizes change, the magnetic and electric traits of epsilon iron oxide change; the researchers began to see unusual electronic behaviour in their samples at larger sizes.

Nickel hopes to continue research on these particles, pursuing some of the stranger magnetic and electric properties.

“The more we are able to investigate these systems and the more we have access to facilities to investigate these systems, the more we can learn about the world around us and develop it into new and transformative technologies,” she says.

This work was funded through the Natural Sciences and Engineering Research Council of Canada and the Canada Foundation for Innovation.

For anyone not familiar with the rare earths situation, they’re not all that rare but they are difficult to mine in most regions of the world. China has some of the most accessible rare earth sites in the world. Consequently, they hold a dominant position in the market. Regardless of who has dominance, this is never a good situation and many countries and their researchers are looking at alternatives to rare earths.

Here’s a link to and a citation for the paper,

Nanoscale Size Effects on Push–Pull Fe–O Hybridization through the Multiferroic Transition of Perovskite ϵ-Fe2O3 by Rachel Nickel, Josh Gibbs, Jacob Burgess, Padraic Shafer, Debora Motta Meira, Chengjun Sun, and Johan van Lierop. Nano Lett. 2023, 23, 17, 7845–7851 DOI: https://doi.org/10.1021/acs.nanolett.3c01512 Publication Date: August 25, 2023 Copyright © 2023 American Chemical Society

This paper is behind a paywall.

NorthPole: a brain-inspired chip design for saving energy

Leave a reply

One of the main attractions of brain-inspired computing is that it requires less energy than is used in conventional computing. The latest entry into the brain-inspired computing stakes was announced in an October 19, 2023 American Association for the Advancement of Science (AAAS) news release on EurekAlert,

Researchers present NorthPole – a brain-inspired chip architecture that blends computation with memory to process data efficiently at low-energy costs. Since its inception, computing has been processor-centric, with memory separated from compute. However, shuttling large amounts of data between memory and compute comes at a high price in terms of both energy consumption and processing bandwidth and speed. This is particularly evident in the case of emerging and advanced real-time artificial intelligence (AI) applications like facial recognition, object detection, and behavior monitoring, which require fast access to vast amounts of data. As a result, most contemporary computer architectures are rapidly reaching physical and processing bottlenecks and risk becoming economically, technically, and environmentally unsustainable, given the growing energy costs involved. Inspired by the neural architecture of the organic brain, Dharmendra Modha and colleagues developed NorthPole – a neural inference architecture that intertwines compute with memory on a single chip. According to the authors, NorthPole “reimagines the interaction between compute and memory” by blending brain-inspired computing and semiconductor technology. It achieves higher performance, energy-efficiency, and area-efficiency compared to other comparable architectures, including those that use more advanced technology processes. And, because NorthPole is a digital system, it is not subject to the device noise and systemic biases and drifts that afflict analog systems. Modha et al. demonstrate NorthPole’s capabilities by testing it on the ResNet50 benchmark image classification network, where it achieved 25 times higher energy metric of frames per second (FPS) per watt, a 5 times higher space metric of FPS per transistor, and a 22 times lower time metric of latency relative to comparable technology. In a related Perspective, Subramanian Iyer and Vwani Roychowdhury discuss NorthPole’s advancements and limitations in greater detail.

By the way, the NorthPole chip is a result of IBM research as noted in Charles Q. Choi’s October 23, 2023 article for IEEE Spectrum magazine (IEEE is the Institute of Electrical and Electronics Engineers), Note: Links have been removed,

A brain-inspired chip from IBM, dubbed NorthPole, is more than 20 times as fast as—and roughly 25 times as energy efficient as—any microchip currently on the market when it comes to artificial intelligence tasks. According to a study from IBM, applications for the new silicon chip may include autonomous vehicles and robotics.

Brain-inspired computer hardware aims to mimic a human brain’s exceptional ability to rapidly perform computations in an extraordinarily energy-efficient manner. These machines are often used to implement neural networks, which similarly imitate the way a brain learns and operates.

“The brain is vastly more energy-efficient than modern computers, in part because it stores memory with compute in every neuron,” says study lead author Dharmendra Modha, IBM’s chief scientist for brain-inspired computing.

“NorthPole merges the boundaries between brain-inspired computing and silicon-optimized computing, between compute and memory, between hardware and software,” Modha says.

…

The scientists note that IBM fabricated NorthPole with a 12-nm node process. The current state of the art for CPUs is 3 nm, and IBM has spent years researching 2-nm nodes. This suggests further gains with this brain-inspired strategy may prove readily available, the company says.

…

The NorthPole chip is preceded by another IBM brain-inspired chip, TrueNorth. (Use the term “TrueNorth” in the blog search engine, if you want to see more about that and other brain-inspired chips.)

Choi’s October 23, 2023 article features technical information but a surprising amount is accessible to an interested reader who’s not an engineer.

There’s a video, which seems to have been produced by IBM,

Here’s a link to and a citation for the paper,

Neural inference at the frontier of energy, space, and time by Dharmendra S. Modha, Filipp Akopyan, Alexander Andreopoulos, Rathinakumar Appuswamy, John V. Arthur, Andrew S. Cassidy, Pallab Datta, Michael V. DeBole, Steven K. Esser, Carlos Ortega Otero, Jun Sawada, Brian Taba, Arnon Amir, Deepika Bablani, Peter J. Carlson, Myron D. Flickner, Rajamohan Gandhasri, Guillaume J. Garreau, Megumi Ito, Jennifer L. Klamo, Jeffrey A. Kusnitz, Nathaniel J. McClatchey, Jeffrey L. McKinstry, Yutaka Nakamura, Tapan K. Nayak, William P. Risk, Kai Schleupen, Ben Shaw, Jay Sivagnaname, Daniel F. Smith, Ignacio Terrizzano, and Takanori Ueda. Science 19 Oct 2023 Vol 382, Issue 6668 pp. 329-335 DOI: 10.1126/science.adh1174

This paper is behind a paywall.

Nanoscale tattoos for individual cells

Leave a reply

It’s fascinating to read about a technique for applying ‘tattoos’ to living cells and I have two news items and news releases with different perspectives about this same research.

First out the door was the August 7, 2023 news item on ScienceDaily,

Engineers have developed nanoscale tattoos — dots and wires that adhere to live cells — in a breakthrough that puts researchers one step closer to tracking the health of individual cells.

The new technology allows for the first time the placement of optical elements or electronics on live cells with tattoo-like arrays that stick on cells while flexing and conforming to the cells’ wet and fluid outer structure.

“If you imagine where this is all going in the future, we would like to have sensors to remotely monitor and control the state of individual cells and the environment surrounding those cells in real time,” said David Gracias, a professor of chemical and biomolecular engineering at Johns Hopkins University who led the development of the technology. “If we had technologies to track the health of isolated cells, we could maybe diagnose and treat diseases much earlier and not wait until the entire organ is damaged.”

…

An August 7, 2023 Johns Hopkins University news release by (also on EurekAlert), which originated the news item, describes the research in an accessible fashion before delving into technical details,

Gracias, who works on developing  biosensor technologies that are nontoxic and noninvasive for the body, said the tattoos bridge the gap between living cells or tissue and conventional sensors and electronic materials. They’re essentially like barcodes or QR codes, he said.

“We’re talking about putting something like an electronic tattoo on a living object tens of times smaller than the head of a pin,” Gracias said. “It’s the first step towards attaching sensors and electronics on live cells.”

The structures were able to stick to soft cells for 16 hours even as the cells moved.

The researchers built the tattoos in the form of arrays with gold, a material known for its ability to prevent signal loss or distortion in electronic wiring. They attached the arrays to cells that make and sustain tissue in the human body, called fibroblasts. The arrays were then treated with  molecular glues and transferred onto the cells using an alginate hydrogel film, a gel-like laminate that can be dissolved after the gold adheres to the cell. The molecular glue on the array bonds to a film secreted by the cells called the extracellular matrix.

Previous research has demonstrated how to use hydrogels to stick nanotechnology onto human skin and internal animal organs. By showing how to adhere nanowires and nanodots onto single cells, Gracias’ team is addressing the long-standing challenge of making optical sensors and electronics compatible with biological matter at the single cell level. 

“We’ve shown we can attach complex nanopatterns to living cells, while ensuring that the cell doesn’t die,” Gracias said. “It’s a very important result that the cells can live and move with the tattoos because there’s often a significant incompatibility between living cells and the methods engineers use to fabricate electronics.”

The team’s ability to attach the dots and wires in an array form is also crucial. To use this technology to track bioinformation, researchers must be able to arrange sensors and wiring into specific patterns not unlike how they are arranged in electronic chips. 

“This is an array with specific spacing,” Gracias explained, “not a haphazard bunch of dots.”

The team plans to try to attach more complex nanocircuits that can stay in place for longer periods. They also want to experiment with different types of cells.

Other Johns Hopkins authors are Kam Sang Kwok, Yi Zuo, Soo Jin Choi, Gayatri J. Pahapale, and Luo Gu.

This looks more like a sea creature to me but it’s not,

Caption: False-colored gold nanodot array on a fibroblast cell. Credit: Kam Sang Kwok and Soo Jin Choi, Gracias Lab/Johns Hopkins University.[The measurement, i.e., what looks like a ‘u’ with a preceding tail, in the lower right corner of the image is one micron/one millionth add that to the ‘m’ and you have what’s commonly described as one micrometre.]

An August 10, 2023 news item on ScienceDaily offers a different perspective from the American Chemical Society (ACS) on this research,

For now, cyborgs exist only in fiction, but the concept is becoming more plausible as science progresses. And now, researchers are reporting in ACS’ Nano Letters that they have developed a proof-of-concept technique to “tattoo” living cells and tissues with flexible arrays of gold nanodots and nanowires. With further refinement, this method could eventually be used to integrate smart devices with living tissue for biomedical applications, such as bionics and biosensing.

…

An August 10, 2023 ACS news release (also on EurekAlert), which originated the news item, explains some of the issues with attaching electronics to living tissue,

Advances in electronics have enabled manufacturers to make integrated circuits and sensors with nanoscale resolution. More recently, laser printing and other techniques have made it possible to assemble flexible devices that can mold to curved surfaces. But these processes often use harsh chemicals, high temperatures or pressure extremes that are incompatible with living cells. Other methods are too slow or have poor spatial resolution. To avoid these drawbacks, David Gracias, Luo Gu and colleagues wanted to develop a nontoxic, high-resolution, lithographic method to attach nanomaterials to living tissue and cells.

The team used nanoimprint lithography to print a pattern of nanoscale gold lines or dots on a polymer-coated silicon wafer. The polymer was then dissolved to free the gold nanoarray so it could be transferred to a thin piece of glass. Next, the gold was functionalized with cysteamine and covered with a hydrogel layer, which, when peeled away, removed the array from the glass. The patterned side of this flexible array/hydrogel layer was coated with gelatin and attached to individual live fibroblast cells. In the final step, the hydrogel was degraded to expose the gold pattern on the surface of the cells. The researchers used similar techniques to apply gold nanoarrays to sheets of fibroblasts or to rat brains. Experiments showed that the arrays were biocompatible and could guide cell orientation and migration.

The researchers say their cost-effective approach could be used to attach other nanoscale components, such as electrodes, antennas and circuits, to hydrogels or living organisms, thereby opening up opportunities for the development of biohybrid materials, bionic devices and biosensors.

The authors acknowledge funding from the Air Force Office of Scientific Research, the National Institute on Aging, the National Science Foundation and the Johns Hopkins University Surpass Program.

Here’s a link to and a citation for the paper,

Toward Single Cell Tattoos: Biotransfer Printing of Lithographic Gold Nanopatterns on Live Cells by Kam Sang Kwok, Yi Zuo, Soo Jin Choi, Gayatri J. Pahapale, Luo Gu, and David H. Gracias. Nano Lett. 2023, 23, 16, 7477–7484 DOI: https://doi.org/10.1021/acs.nanolett.3c01960 Publication Date:August 1, 2023 Copyright © 2023 American Chemical Society

This paper is behind a paywall.

100-fold increase in AI energy efficiency

Leave a reply

Most people don’t realize how much energy computing, streaming video, and other technologies consume and AI (artificial intelligence) consumes a lot. (For more about work being done in this area, there’s my October 13, 2023 posting about an upcoming ArtSci Salon event in Toronto featuring Laura U. Marks’s recent work ‘Streaming Carbon Footprint’ and my October 16, 2023 posting about how much water is used for AI.)

So this news is welcome, from an October 12, 2023 Northwestern University news release (also received via email and on EurekAlert), Note: Links have been removed,

AI just got 100-fold more energy efficient

Nanoelectronic device performs real-time AI classification without relying on the cloud

– AI is so energy hungry that most data analysis must be performed in the cloud
– New energy-efficient device enables AI tasks to be performed within wearables
– This allows real-time analysis and diagnostics for faster medical interventions
– Researchers tested the device by classifying 10,000 electrocardiogram samples
– The device successfully identified six types of heart beats with 95% accuracy

Northwestern University engineers have developed a new nanoelectronic device that can perform accurate machine-learning classification tasks in the most energy-efficient manner yet. Using 100-fold less energy than current technologies, the device can crunch large amounts of data and perform artificial intelligence (AI) tasks in real time without beaming data to the cloud for analysis.

With its tiny footprint, ultra-low power consumption and lack of lag time to receive analyses, the device is ideal for direct incorporation into wearable electronics (like smart watches and fitness trackers) for real-time data processing and near-instant diagnostics.

To test the concept, engineers used the device to classify large amounts of information from publicly available electrocardiogram (ECG) datasets. Not only could the device efficiently and correctly identify an irregular heartbeat, it also was able to determine the arrhythmia subtype from among six different categories with near 95% accuracy.

The research was published today (Oct. 12 [2023]) in the journal Nature Electronics.

“Today, most sensors collect data and then send it to the cloud, where the analysis occurs on energy-hungry servers before the results are finally sent back to the user,” said Northwestern’s Mark C. Hersam, the study’s senior author. “This approach is incredibly expensive, consumes significant energy and adds a time delay. Our device is so energy efficient that it can be deployed directly in wearable electronics for real-time detection and data processing, enabling more rapid intervention for health emergencies.”

A nanotechnology expert, Hersam is Walter P. Murphy Professor of Materials Science and Engineering at Northwestern’s McCormick School of Engineering. He also is chair of the Department of Materials Science and Engineering, director of the Materials Research Science and Engineering Center and member of the International Institute of Nanotechnology. Hersam co-led the research with Han Wang, a professor at the University of Southern California, and Vinod Sangwan, a research assistant professor at Northwestern.

Before machine-learning tools can analyze new data, these tools must first accurately and reliably sort training data into various categories. For example, if a tool is sorting photos by color, then it needs to recognize which photos are red, yellow or blue in order to accurately classify them. An easy chore for a human, yes, but a complicated — and energy-hungry — job for a machine.

For current silicon-based technologies to categorize data from large sets like ECGs, it takes more than 100 transistors — each requiring its own energy to run. But Northwestern’s nanoelectronic device can perform the same machine-learning classification with just two devices. By reducing the number of devices, the researchers drastically reduced power consumption and developed a much smaller device that can be integrated into a standard wearable gadget.

The secret behind the novel device is its unprecedented tunability, which arises from a mix of materials. While traditional technologies use silicon, the researchers constructed the miniaturized transistors from two-dimensional molybdenum disulfide and one-dimensional carbon nanotubes. So instead of needing many silicon transistors — one for each step of data processing — the reconfigurable transistors are dynamic enough to switch among various steps.

“The integration of two disparate materials into one device allows us to strongly modulate the current flow with applied voltages, enabling dynamic reconfigurability,” Hersam said. “Having a high degree of tunability in a single device allows us to perform sophisticated classification algorithms with a small footprint and low energy consumption.”

To test the device, the researchers looked to publicly available medical datasets. They first trained the device to interpret data from ECGs, a task that typically requires significant time from trained health care workers. Then, they asked the device to classify six types of heart beats: normal, atrial premature beat, premature ventricular contraction, paced beat, left bundle branch block beat and right bundle branch block beat.

The nanoelectronic device was able to identify accurately each arrhythmia type out of 10,000 ECG samples. By bypassing the need to send data to the cloud, the device not only saves critical time for a patient but also protects privacy.

“Every time data are passed around, it increases the likelihood of the data being stolen,” Hersam said. “If personal health data is processed locally — such as on your wrist in your watch — that presents a much lower security risk. In this manner, our device improves privacy and reduces the risk of a breach.”

Hersam imagines that, eventually, these nanoelectronic devices could be incorporated into everyday wearables, personalized to each user’s health profile for real-time applications. They would enable people to make the most of the data they already collect without sapping power.

“Artificial intelligence tools are consuming an increasing fraction of the power grid,” Hersam said. “It is an unsustainable path if we continue relying on conventional computer hardware.”

…

Here’s a link to and a citation for the paper,

Reconfigurable mixed-kernel heterojunction transistors for personalized support vector machine classification by Xiaodong Yan, Justin H. Qian, Jiahui Ma, Aoyang Zhang, Stephanie E. Liu, Matthew P. Bland, Kevin J. Liu, Xuechun Wang, Vinod K. Sangwan, Han Wang & Mark C. Hersam. Nature Electronics (2023) DOI: https://doi.org/10.1038/s41928-023-01042-7 Published: 12 October 2023

This paper is behind a paywall.