Tag Archives: Nitin Dahad

Neuromorphic (brainlike) computing and your car (a Mercedes Benz Vision AVTR concept car)

If you’ve ever fantasized about a batmobile of your own, the dream could come true soon,

Mercedes Berz VISION AVTR [downloaded from https://www.mercedes-benz.com/en/innovation/concept-cars/vision-avtr/]

It was the mention of neuromorphic computing in a television ad sometime in September 2022 that sent me on a mission to find out what Mercedes Benz means when they use neuromorphic computing to describe a feature found in their Vision AVTR concept car. First, a little bit about the car (from the Vision AVTR webpage accessed in October 2022),

VISION AVTR – inspired by AVATAR.

The name of the groundbreaking concept vehicle stands not only for the close collaboration in developing the show car together with the AVATAR team but also for ADVANCED VEHICLE TRANSFORMATION. This concept vehicle embodies the vision of Mercedes-Benz designers, engineers and trend researchers for mobility in the distant future.

,,,

Organic battery technology.

The VISION AVTR was designed in line with its innovative electric drive. This is based on a particularly powerful and compact high-voltage battery. For the first time, the revolutionary battery technology is based on graphene-based [emphasis mine] organic cell chemistry and thus completely eliminates rare, toxic and expensive earths such as metals. Electromobility thus becomes independent of fossil resources. An absolute revolution is also the recyclability by composting, which is 100% recyclable due to the materiality. As a result, Mercedes-Benz underlines the high relevance of a future circular economy in the raw materials sector.

Masterpiece of efficiency.

At Mercedes-Benz, the consideration of efficiency goes far beyond the drive concept, because with increasing digitalisation, the performance of the large number of so-called secondary consumers also comes into focus – along with their efficient energy supply, without negatively affecting the drive power of the vehicle itself. Energy consumption per computing operation is already a key target in the development of new computer chips. This trend will continue in the coming years with the growth of sensors and artificial intelligence in the automotive industry. The neuro-inspired approach of the VISION AVTR, including so-called neuromorphic hardware, promises to minimise the energy requirements of sensors, chips and other components to a few watts. [emphasis mine] Their energy supply is provided by the cached current of the integrated solar plates on the back of the VISION AVTR. The 33 multi-directionally movable surface elements act as “bionic flaps”.

Interior and exterior merge.

For the first time, Mercedes-Benz has worked with a completely new design approach in the design of the VISION AVTR. The holistic concept combines the design disciplines interior, exterior and UX [user experience] from the first sketch. Man and human perception are the starting point of a design process from the inside out. The design process begins with the experience of the passengers and consciously focuses on the perception and needs of the passengers. The goal was to create a car that prolongs the perception of its passengers. It was also a matter of creating an immersive experience space in which passengers connect with each other, with the vehicle and the surrounding area [emphasis mine ] in a unique way.

Intuitive control.

The VISION AVTR already responds to the approach of the passengers by visualising the energy and information flow of the environment with digital neurons that flow through the grille through the wheels to the rear area. The first interaction in the interior between man and vehicle happens completely intuitively via the control unit: by placing the hand on the centre console, the interior comes to life and the vehicle recognises the driver by his breathing. This is made visible on the instrument panel and on the user’s hand. The VISION AVTR thus establishes a biometric connection with the driver [emphasis mine] and increases his awareness of the environment. The digital neurons flow from the interior into the exterior and visualise the flow of energy and information. For example, when driving, the neurons flow over the outside of the vehicle. [emphasis mine] When changing direction, the energy flows to the corresponding side of the vehicle.

The vehicle as an immersive experience space.

The visual connection between passengers and the outside world is created by the curved display module, which replaces a conventional dashboard. The outside world around the vehicle and the surrounding area is shown in real-time 3D graphics and at the same time shows what is happening on the road in front of the vehicle. Combined with energy lines, these detailed real-time images bring the interior to life and allow passengers to discover and interact with the environment in a natural way with different views of the outside world. Three wonders of nature – the Huangshan Mountains of China, the 115-metre-high Hyperion Tree found in the United States and the pink salt Lake Hillier from Australia – can be explored in detail. Passengers become aware of various forces of nature that are not normally visible to the human eye, such as magnetic fields, bioenergy or ultraviolet light.

The curved display module in the Mercedes-Benz VISION AVTR – inspired by AVATAR
[downloaded from https://www.mercedes-benz.com/en/innovation/concept-cars/vision-avtr/]

Bionic formal language.

When the boundaries between vehicle and living beings are lifted, Mercedes-Benz combines luxury and sustainability and works to make the vehicles as resource-saving as possible. With the VISION AVTR, the brand is now showing how a vehicle can blend harmoniously into its environment and communicate with it. In the ecosystem of the future, the ultimate luxury is the fusion of human and nature with the help of technology. The VISION AVTR is thus an example of sustainable luxury in the field of design. As soon as you get in, the car becomes an extension of your own body and a tool to discover the environment much as in the film humans can use avatars to extend and expand their abilities.

A few thoughts

The movie, Avatar, was released in 2009 and recently rereleased in movie houses in anticipation of the sequel, Avatar: The Way of Water to be released in December 2022 (Avatar [2009 film] Wikipedia entry). The timing, Avatar and AVTR, is interesting, oui?

Moving onto ‘organic’, which means carbon-based in this instance and, specifically, graphene. Commercialization of graphene is likely top-of-mind for the folks (European Commission) who bet 1B Euros in 2013 with European Union money to fund the Graphene Flagship project. This battery from German company Mercedes Benz must be exciting news for the funders and for people who want to lessen dependency on rare earths. Your battery can be composted safely (according to the advertising).

The other piece of good news, is the neuromorphic computing,

“The neuro-inspired approach of the VISION AVTR, including so-called neuromorphic hardware, promises to minimise the energy requirements of sensors, chips and other components to a few watts.”

On the other hand and keeping in mind the image above (a hand with what looks like an embedded object), it seems a little disconcerting to merge with one’s car, “… passengers connect with each other, with the vehicle and the surrounding area …” which becomes even more disconcerting when this appears in the advertising,

… VISION AVTR thus establishes a biometric connection with the driver … The digital neurons flow from the interior into the exterior and visualise the flow of energy and information. For example, when driving, the neurons flow over the outside of the vehicle.

Are these ‘digital neurons’ flowing around the car like a water current? Also, the car is visualizing? Hmm …

I did manage to find a bit more information about neuromorphic computing although it’s for a different Mercedes Benz concept car (there’s no mention of flowing digital neurons) in a January 18, 2022 article by Sally Ward-Foxton for EE Times (Note: A link has been removed),

The Mercedes Vision EQXX concept car, promoted as “the most efficient Mercedes-Benz ever built,” incorporates neuromorphic computing to help reduce power consumption and extend vehicle range. To that end, BrainChip’s Akida neuromorphic chip enables in-cabin keyword spotting as a more power-efficient way than existing AI-based keyword detection systems.

“Working with California-based artificial intelligence experts BrainChip, Mercedes-Benz engineers developed systems based on BrainChip’s Akida hardware and software,” Mercedes noted in a statement describing the Vision EQXX. “The example in the Vision EQXX is the “Hey Mercedes” hot-word detection. Structured along neuromorphic principles, it is five to ten times more efficient than conventional voice control,” the carmaker claimed.

That represents validation of BrainChip’s technology by one of its early-access customers. BrainChip’s Akida chip accelerates spiking neural networks (SNNs) and convolutional neural networks (via conversion to SNNs). It is not limited to a particular application, and also run [sic] person detection, voice or face recognition SNNs, for example, that Mercedes could also explore.

This January 6, 2022 article by Nitin Dahad for embedded.com describes what were then the latest software innovations in the automotive industry and segues into a description of spiking neural networks (Note: A link has been removed),

The electric vehicle (EV) has clearly become a key topic of discussion, with EV range probably the thing most consumers are probably worried about. To address the range concern, two stories emerged this week – one was Mercedes-Benz’ achieving a 1,000 km range with its VISION EQXX prototype, albeit as a concept car, and General Motors announcing during a CES [Consumer Electronics Show] 2022 keynote its new Chevrolet Silverado EV with 400-mile (640 km) range.

In briefings with companies, I often hear them talk about the software-defined car and the extensive use of software simulation (or we could also call it a digital twin). In the case of both the VISION EQXX and the Silverado EV, software plays a key part. I also spoke to BlackBerry about its IVY platform and how it is laying the groundwork for software-defined vehicles.

Neuromorphic computing for infotainment

This efficiency is not just being applied to enhancing range though. Mercedes-Benz also points out that its infotainment system uses neuromorphic computing to enable the car to take to “take its cue from the way nature thinks”.

Mercedes-Benz VISION EQXXMercedes-Benz VISION EQXX

The hardware runs spiking neural networks, in which data is coded in discrete spikes and energy only consumed when a spike occurs, reducing energy consumption by orders of magnitude. In order to deliver this, the carmaker worked with BrainChip, developing the systems based on its Akida processor. In the VISION EQXX, this technology enables the “Hey Mercedes” hot-word detection five to ten times more efficiently than conventional voice control. Mercedes-Benz said although neuromorphic computing is still in its infancy, systems like these will be available on the market in just a few years. When applied on scale throughout a vehicle, they have the potential to radically reduce the energy needed to run the latest AI technologies.

For anyone curious about BrainChip, you can find out more here.

It took a little longer than I hoped but I’m glad that I found out a little more about neuromorphic computing and one application in the automotive industry.