Tag Archives: University of California at Merced

What human speech, jazz, and whale song have in common

Credit: iStock/Velvetfish

Seeing connections between what seem to be unrelated activities such as human speech, jazz, and whale song is fascinating to me and I’m not alone. Scientists at the University of California at Merced (UC Merced) have delivered handily on that premise according to an Oct. 13, 2017 news item on phys.org,

Jazz musicians riffing with each other, humans talking to each other and pods of killer whales all have interactive conversations that are remarkably similar to each other, new research reveals.

Cognitive science researchers at UC Merced have developed a new method for analyzing and comparing the sounds of speech, music and complex animal vocalizations like whale song and bird song. The paper detailing their findings is being published today [Oct. 12, 2017] in the Journal of the Royal Society Interface.

Their method is based on the idea that these sounds are complex because they have multiple layers of structure. Every language, for instance, has individuals sounds, roughly corresponding to letters, that combine to form syllables, words, phrases, sentences and so on. It’s a hierarchy that everyone understands intuitively. Musical compositions have their own temporal hierarchies, but until now there hasn’t been a way to directly compare the hierarchies of speech and music, or test whether similar hierarchies might exist in bird song and whale song.

An Oct. 12, 2017 UC Merced news release by Lorena Anderson, which originated the news item, provides more details about the investigation (Note: Links have been removed),

“Playing jazz music has been likened to a conversation among musicians, and killer whales are highly social creatures who vocalize as if they are talking to each other. But does jazz music really sound like a conversation, and do killer whales really sound like they are talking?” asked lead researcher and UC Merced professor Chris Kello. “We know killer whales are highly social and intelligent, but it’s hard to tell that they are interacting when you listen to recordings of them. Our method shows how much their sound patterns are like people talking, but not like other, less social whales or birds.”

The researchers figured out a way to measure and compare sound recordings by converting them into “barcodes” that capture clusters of sound energy, and clusters of clusters, across levels of a hierarchy. These barcodes allowed the researchers to directly compare temporal hierarchies in more than 200 recordings of different kinds of speech in six different languages, different kinds of popular and classical music, four different species of birds and whales singing their songs, and even thunderstorms.

Kello and his colleagues have been using the barcode method for several years. They first developed it in studies of conversations. The study published today is the first time that they applied the method to music and animal vocalizations.

“The method allows us to ask questions about language and music and animal songs that we couldn’t ask without a way to see and compare patterns in all these recordings,” Kello said.

A common song

The researchers compared barcode-style visualizations of recorded sounds.
Credit: UC Merced

Kello, fellow UC Merced cognitive science professor Ramesh Balasubramaniam, graduate student Butovens Me´de´ [or Médé] and collaborator professor Simone Dalla Bella also discovered that the haunting songs of huge humpback whales are remarkably similar to the beautiful songs of tiny nightingales and hermit thrushes in terms of their temporal hierarchies.

“Humpbacks, nightingales and hermit thrushes are solitary singers,” Kello said. “The barcodes show that their songs have similar layers of structure, but we don’t know what it means — yet.”

The idea for this project came from Kello’s sabbatical at the University of Montpellier in France, where he worked and discussed ideas with Dalla Bella. Balasubramaniam, who studies how music is perceived, is in the School of Social Sciences, Humanities and Arts with Kello, who studies speech and language processing. The project was a natural collaboration and is part of a growing research focus at UC Merced that was enabled by the National Science Foundation-funded CHASE summer school on Music and Language in 2014, and a Google Faculty Award to Kello.

Balasubramaniam is interested in continuing the work to better understand how brains distinguish between music and speech, while Kello said there are many different avenues to pursue.

For instance, the researchers found nearly identical temporal hierarchies for six different languages, which may suggest something universal about human speech. However, because this result was based on recordings of TED Talks — which have a common style and progression — Kello said it will be important to keep looking at other forms of speech and language.

One of his graduate students, Sara Schneider, is using the method to study the convergence of Spanish and English barcodes in bilingual conversations. Another graduate student, Adolfo Ramirez-Aristizabal, is working with Kello and Balasubramaniam to study whether the barcode method may shed light on how brains process speech and other complex sounds.

“Listening to music and speech, we can hear some of what we see in the barcodes, and the information may be useful for automatic classification of audio recordings. But that doesn’t mean that our brains process music and speech using these barcodes,” Kello said. “It’s intriguing, but we need to keep asking questions and go where the data lead us.”

Here’s a link to and a citation for the paper,

Hierarchical temporal structure in music, speech and animal vocalizations: jazz is like a conversation, humpbacks sing like hermit thrushes by Christopher T. Kello, Simone Dalla Bella, Butovens Médé, Ramesh Balasubramaniam. Journal of the Royal Society Interface DOI: 10.1098/rsif.2017.0231 Published 11 October 2017

This paper appears to be open access.*

*”This paper is behind a paywall” was changed to “… appears to be open access.” at 1700 hours on January 23, 2018.

Dr. Frankenstein and competitive exclusion

A promotional photo of Boris Karloff as Frankenstein's monster, using Jack Pierce's makeup design. Credit:: Universal Studios

A promotional photo of Boris Karloff as Frankenstein’s monster, using Jack Pierce’s makeup design. Credit:: Universal Studios

An Oct. 28, 2016 news item on phys.org provides some new insight into the ‘Frankenstein story’ and its perspective on science,

Frankenstein as we know him, the grotesque monster that was created through a weird science experiment, is actually a nameless Creature created by scientist Victor Frankenstein in Mary Shelley’s 1818 novel, “Frankenstein.” Widely considered the first work of science fiction for exploring the destructive consequences of scientific and moral transgressions, a new study published in BioScience argues that the horror of Mary Shelley’s gothic novel is rooted in a fundamental principle of biology.

The co-authors point to a pivotal scene when the Creature encounters Victor Frankenstein and requests a female companion to mitigate his loneliness. The Creature distinguishes his dietary needs from those of humans and expresses a willingness to inhabit the “wilds of South America,” suggesting distinct ecological requirements. Frankenstein concedes to this reasoning given that humans would have few competitive interactions with a pair of isolated creatures, but he then reverses his decision after considering the creatures’ reproductive potential and the probability of human extinction, a concept termed competitive exclusion. In essence, Frankenstein was saving humankind.

An Oct. 28, 2016 Dartmouth College news release (also on EurekAlert) by Amy Olson, which originated the news item, describes the co-authors and the research in more detail (Note: Links have been removed),

A study co-authored by Dartmouth’s Nathaniel Dominy casts a new light on the story of Frankenstein’s monster, who lives on in the public imagination in stories, in movies, and of course, on Halloween.

Mary Shelley’s gothic novel is rooted in a fundamental principle of biology, and its horror lies in the specter of the extinction of the human race, say Dominy, a professor of anthropology, and his coauthor, Justin Yeakel.

“The principle of competitive exclusion was not formally defined until the 1930s,” says Dominy. “Given Shelley’s early command of this foundational concept, we used computational tools developed by ecologists to explore if, and how quickly, an expanding population of creatures would drive humans to extinction.”

The authors developed a mathematical model based on human population densities in 1816, finding that the competitive advantages of creatures varied under different circumstances. The worst-case scenario for humans was a growing population of creatures in South America, as it was a region with fewer humans and therefore less competition for resources.

“We calculated that a founding population of two creatures could drive us to extinction in as little as 4,000 years,” says Dominy. Although the study is merely a thought experiment, it casts new light on the underlying horror of the novel: the extinction of the human race. It also has real-word implications for how we understand the biology of invasive species.

“To date, most scholars have focused on Mary Shelley’s knowledge of then-prevailing views on alchemy, physiology, and resurrection; however, the genius of Mary Shelley lies in how she combined and repackaged existing scientific debates to invent the genre of science fiction,” says Justin D. Yeakel, an Omidyar fellow at the Santa Fe Institute and an assistant professor in the School of Natural Sciences at the University of California, Merced.

“Our study adds to Mary Shelley’s legacy, by showing that her science fiction accurately anticipated fundamental concepts in ecology and evolution by many decades,” he says.

Here’s a link to and a citation for the paper,

Frankenstein and the Horrors of Competitive Exclusion by Nathaniel J. Dominy and Justin D. Yeakel.  BioScience (2016) doi: 10.1093/biosci/biw133 First published online: October 28, 2016

This paper is behind a paywall.