Tag Archives: Domenico Vicinanza

I hear the proteins singing

Points to anyone who recognized the paraphrasing of the title for the well-loved, Canadian movie, “I heard the mermaids singing.” In this case, it’s all about protein folding and data sonification (from an Oct. 20, 2016 news item on phys.org),

Transforming data about the structure of proteins into melodies gives scientists a completely new way of analyzing the molecules that could reveal new insights into how they work – by listening to them. A new study published in the journal Heliyon shows how musical sounds can help scientists analyze data using their ears instead of their eyes.

The researchers, from the University of Tampere in Finland, Eastern Washington University in the US and the Francis Crick Institute in the UK, believe their technique could help scientists identify anomalies in proteins more easily.

An Oct. 20, 2016 Elsevier Publishing press release on EurekAlert, which originated the news item, expands on the theme,

“We are confident that people will eventually listen to data and draw important information from the experiences,” commented Dr. Jonathan Middleton, a composer and music scholar who is based at Eastern Washington University and in residence at the University of Tampere. “The ears might detect more than the eyes, and if the ears are doing some of the work, then the eyes will be free to look at other things.”

Proteins are molecules found in living things that have many different functions. Scientists usually study them visually and using data; with modern microscopy it is possible to directly see the structure of some proteins.

Using a technique called sonification, the researchers can now transform data about proteins into musical sounds, or melodies. They wanted to use this approach to ask three related questions: what can protein data sound like? Are there analytical benefits? And can we hear particular elements or anomalies in the data?

They found that a large proportion of people can recognize links between the melodies and more traditional visuals like models, graphs and tables; it seems hearing these visuals is easier than they expected. The melodies are also pleasant to listen to, encouraging scientists to listen to them more than once and therefore repeatedly analyze the proteins.

The sonifications are created using a combination of Dr. Middleton’s composing skills and algorithms, so that others can use a similar process with their own proteins. The multidisciplinary approach – combining bioinformatics and music informatics – provides a completely new perspective on a complex problem in biology.

“Protein fold assignment is a notoriously tricky area of research in molecular biology,” said Dr. Robert Bywater from the Francis Crick Institute. “One not only needs to identify the fold type but to look for clues as to its many functions. It is not a simple matter to unravel these overlapping messages. Music is seen as an aid towards achieving this unraveling.”

The researchers say their molecular melodies can be used almost immediately in teaching protein science, and after some practice, scientists will be able to use them to discriminate between different protein structures and spot irregularities like mutations.

Proteins are the first stop, but our knowledge of other molecules could also benefit from sonification; one day we may be able to listen to our genomes, and perhaps use this to understand the role of junk DNA [emphasis mine].

About 97% of our DNA (deoxyribonucleic acid) has been known for some decades as ‘junk DNA’. In roughly 2012, that was notion was challenged as Stephen S. Hall wrote in an Oct. 1, 2012 article (Hidden Treasures in Junk DNA; What was once known as junk DNA turns out to hold hidden treasures, says computational biologist Ewan Birney) for Scientific American.

Getting back to  2016, here’s a link to and a citation for ‘protein singing’,

Melody discrimination and protein fold classification by  Robert P. Bywater, Jonathan N. Middleton. Heliyon 20 Oct 2016, Volume 2, Issue 10 DOI: 10.1016/j.heliyon.2016.e0017

This paper is open access.

Here’s what the proteins sound like,

Supplementary Audio 3 for file for Supplementary Figure 2 1r75 OHEL sonification full score. [downloaded from the previously cited Heliyon paper]

Joanna Klein has written an Oct. 21, 2016 article for the New York Times providing a slightly different take on this research (Note: Links have been removed),

“It’s used for the concert hall. It’s used for sports. It’s used for worship. Why can’t we use it for our data?” said Jonathan Middleton, the composer at Eastern Washington University and the University of Tampere in Finland who worked with Dr. Bywater.

Proteins have been around for billions of years, but humans still haven’t come up with a good way to visualize them. Right now scientists can shoot a laser at a crystallized protein (which can distort its shape), measure the patterns it spits out and simulate what that protein looks like. These depictions are difficult to sift through and hard to remember.

“There’s no simple equation like e=mc2,” said Dr. Bywater. “You have to do a lot of spade work to predict a protein structure.”

Dr. Bywater had been interested in assigning sounds to proteins since the 1990s. After hearing a song Dr. Middleton had composed called “Redwood Symphony,” which opens with sounds derived from the tree’s DNA, he asked for his help.

Using a process called sonification (which is the same thing used to assign different ringtones to texts, emails or calls on your cellphone) the team took three proteins and turned their folding shapes — a coil, a turn and a strand — into musical melodies. Each shape was represented by a bunch of numbers, and those numbers were converted into a musical code. A combination of musical sounds represented each shape, resulting in a song of simple patterns that changed with the folds of the protein. Later they played those songs to a group of 38 people together with visuals of the proteins, and asked them to identify similarities and differences between them. The two were surprised that people didn’t really need the visuals to detect changes in the proteins.

Plus, I have more about data sonification in a Feb. 7, 2014 posting regarding a duet based on data from Voyager 1 & 2 spacecraft.

Finally, I hope my next Steep project will include  sonification of data on gold nanoparticles. I will keep you posted on any developments.

Creative destruction for Canada’s fundamental science

After receiving an ‘invitation’ from the Canadian Science Policy Centre, I wrote an opinion piece, drawing on my submission for the public consultation on Canada’s fundamental science research. It seems the invitation was more of a ‘call’ for submissions and my piece did not end up being selected for inclusion on the website. So rather than waste the piece, here it is,

Creative destruction for Canada’s fundamental science

At a time when we are dealing with the consequences of our sins and virtues, fundamental science, at heart, an exercise in imagination, can seem a waste of precious time. Pollution and climate change (sins: ill-considered uses of technology) and food security and water requirements (virtues: efforts to improve health and save more lives) would seem to demand solutions not the flights of fancy associated with basic science. After all, what does the ‘big bang’ have to do with potable water?

It’s not an unfair question despite the impatience some might feel when answering it by citing a number of practical applications which are the result of all that ‘fanciful’ or ‘blue sky’ science. The beauty and importance of the question is that it will always be asked and can never be definitively answered, rendering it a near constant goad or insurance against complacency.

In many ways Canada’s review of fundamental science (deadline for comments was Sept. 30, 2016) is not just an examination of the current funding schemes but an opportunity to introduce more ‘goads’ or ‘anti-complacency’ measures into Canada’s fundamental science efforts for a kind of ‘creative destruction’.

Introduced by economist Joseph Schumpeter, the concept is derived from Karl Marx’s work but these days is associated with disruptive, painful, and regenerative innovation of all kinds and Canadian fundamental science needs more ‘creative destruction’. There’s at least one movement in this direction (found both in Canada and internationally) which takes us beyond uncomfortable, confrontative questions and occasional funding reviews—the integration of arts and humanities as an attempt at ‘creative destruction’ of the science endeavour.

At one point in the early 2000s, Canada developed a programme where the National Research Council could get joint funding with the Canada Council for the Arts for artists to work with their scientists. It was abandoned a few years later, as a failure. But, since then, several informal attempts at combining arts, sciences, and humanities have sprung up.

For example, Curiosity Collider (founded in 2015) hosts artists and scientists presenting their art/science pieces at various events in Vancouver. Beakerhead has mashed up science, engineering, arts, and entertainment in a festival founded and held in Calgary since 2013. Toronto’s ArtSci Salon hosts events and installations for local, national, and international collaborations of artists and scientists. And, getting back to Vancouver, Anecdotal Evidence is a science storytelling series which has been appearing sporadically since 2015.

There is a tendency to dismiss these types of collaboration as a form of science outreach designed to amuse or entertain but they can be much more than that. Illustrators have taught botanists a thing or two about plants. Markus Buehler at the Massachusetts Institute of Technology has used his understanding of music to explore material science (spider’s webs). Domenico Vicinanza has sonified data from space vehicle, Voyager 1, to produce a symphony, which is also a highly compressed means of communicating data.

C. P. Snow’s ‘The Two Cultures’ (lecture and book) covered much of the same territory in 1959 noting the idea that the arts and sciences (and humanities) can and should be linked in some fashion was not new. For centuries the sciences were referred to as Natural Philosophy (humanities), albeit only chemistry and physics were considered sciences, and many universities have or had faculties of arts and sciences or colleges of arts and science (e.g., the University of Saskatchewan still has such a college).

The current art/sci or sci-art movement can be seen as more than an attempt to resuscitate a ‘golden’ period from the past. It could be a means of embedding a continuous state of regeneration or ‘creative destruction’ for fundamental science in Canada.

Data sonification: listening to your data instead of visualizing it

Representing data though music is how a Jan. 31, 2014 item on the BBC news magazine describes a Voyager 1 & 2 spacecraft duet, data sonification project discussed* in a BBC Radio 4 programme,

Musician and physicist Domenico Vicinanza has described to BBC Radio 4’s Today programme the process of representing information through music, known as “sonification”. [includes a sound clip and interview with Vicinanza]

A Jan. 22, 2014 GÉANT news release describes the project in more detail,

GÉANT, the pan-European data network serving 50 million research and education users at speeds of up to 500Gbps, recently demonstrated its power by sonifying 36 years’ worth of NASA Voyager spacecraft data and converting it into a musical duet.

The project is the work of Domenico Vicinanza, Network Services Product Manager at GÉANT. As a trained musician with a PhD in Physics, he also takes the role of Arts and Humanities Manager, exploring new ways for representing data and discovery through the use of high-speed networks.

“I wanted to compose a musical piece celebrating the Voyager 1 and 2 *together*, so used the same measurements (proton counts from the cosmic ray detector over the last 37 years) from both spacecrafts, at the exactly same point of time, but at several billions of Kms of distance one from the other.

I used different groups of instruments and different sound textures to represent the two spacecrafts, synchronising the measurements taken at the same time.”

The result is an up-tempo string and piano orchestral piece.

You can hear the duet, which has been made available by the folks at GÉANT,

The news release goes on to provide technical details about the composition,

To compose the spacecraft duet, 320,000 measurements were first selected from each spacecraft, at one hour intervals. Then that data was converted into two very long melodies, each comprising 320,000 notes using different sampling frequencies, from a few KHz to 44.1 kHz.

The result of the conversion into waveform, using such a big dataset, created a wide collection of audible sounds, lasting just a few seconds (slightly more than 7 seconds at 44.1kHz) to a few hours (more than 5hours using 1024Hz as a sampling frequency).   A certain number of data points, from a few thousand to 44,100 were each “converted” into 1 second of sound.

Using the grid computing facilities at EGI, GÉANT was able to create the duet live at the NASA booth at Super Computing 2013 using its superfast network to transfer data to/from NASA.

I think this detail from the news release gives one a different perspective on the accomplishment,

Launched in 1977, both Voyager 1 and Voyager 2 are now decommissioned but still recording and sending live data to Earth. They continue to traverse different parts of the universe, billions of kilometres apart. Voyager 1 left our solar system last year.

The research is more than an amusing way to pass the time (from the news release),

While this project was created as a fun, accessible way to demonstrate the benefit of research and education networks to society, data sonification – representing data by means of sound signals – is increasingly used to accelerate scientific discovery; from epilepsy research to deep space discovery.

I was curious to learn more about how data represented by sound signals is being used to accelerate scientific discovery and sent that question and another to Dr. Vicinanza via Tamsin Henderson of DANTE and received these answers,

(1) How does “representing data by means of sound signals “increasingly accelerate scientific discovery; from epilepsy research to deep space discovery”? In a practical sense how does one do this research? For example, do you sit down and listen to a file and intuit different relationships for the data?

Vision and visual representation is intrinsically limited to three dimensions. We all know how amazing is 3D cinema, but in terms of representation of complex information, this is as far as it gets. There is no 4D or 5D. We live in three dimensions.

Sound, on the other hand, does not have any limitation of this kind. We can continue overlapping sound layers virtually without limits and still retain the capability of recognising and understanding them. Think of an orchestra or a pop band, even if the musicians are playing all together we can actually follow the single instrument line (bass, drum, lead guitar, voice, ….) Sound is then particularly precious when dealing with multi-dimensional data since audification techniques.

In technical terms, auditory perception of complex, structured information could have several advantages in temporal, amplitude, and frequency resolution when compared to visual representations and often opens up possibilities as an alternative or complement to visualisation techniques. Those advantages include the capability of the human ear to detect patterns (detecting regularities), recognise timbres and follow different strands at the same time (i.e. the capability of following different instrument lines). This would offer, in a natural way, the opportunity of rendering different, interdependent variables onto sounds in such a way that a listener could gain relevant insight into the represented information or data.

In particular in the medical context, there have been several investigations using data sonification as a support tool for classification and diagnosis, from working on sonification of medical images to converting EEG to tones, including real-time screening and feedback on EEG signals for epilepsy.

The idea is to use sound to aggregate many “information layers”, many more than any graph or picture can represent and support the physician giving a more comprehensive representation of the situation.

(2) I understand that as you age certain sounds disappear from your hearing, e.g., people over 25 years of age are not be able to hear above 15kHz. (Note: There seems to be some debate as to when these sounds disappear, after 30, after 20, etc.) Wouldn’t this pose an age restriction on the people who could access the research or have I misunderstood what you’re doing?

No, there is actually no sensible reduction in the advantages of sonification with ageing. The only precaution is not to use too high frequencies (above 15 KHz) in the sonification and this is something that can be avoided without limiting the benefits of audification.

It is always good practice not to use excessively high frequencies since they are not always very well and uniformly perceived by everyone.

Our hearing works at its best in the region of KHz (1200Hz-3800Hz)

Thank you Dr. Vicinanza and Tamsin Henderson for this insight into representing data in multiple dimensions using sound and its application in research. And, thank you, too, for sharing a beautiful piece of music.

For the curious, I found some additional information about Dr. Vicinanza and his ‘sound’ work on his Nature Network profile page,

I am a composer, network engineer and researcher. I received my MSc and PhD degrees in Physics and studied piano, percussion and composition.

I worked as a professor of Sound Synthesis, Acoustics and Computer Music (Algorithmic Composition) at Conservatory of Music of Salerno (Italy).

I currently work as a network engineer in DANTE (www.dante.net) and chair the ASTRA project (www.astraproject.org) for the reconstruction of musical instruments by means of computer models on GÉANT and EUMEDCONNECT.

I am also the co-founder and the technical coordinator of the Lost Sound Orchestra project (www.lostsoundsorchestra.org).

Interests

As a composer and researcher I was always fascinated by the richness of the information coming from the Nature. I worked on the introduction of the sonification of seismic signals (in particular coming from active volcanoes) as a scientific tool, co-working with geophysicists and volcanologists.

I also study applications of grid technologies for music and visual arts and as a composer I took part to several concerts, digital arts performances, festivals and webcast.

My other interests include (aside with music) Argentine Tango and watercolors.

Projects

ASTRA (Ancient instruments Sound/Timbre Reconstruction Application)
www.astraproject.org

The ASTRA project is a multi disciplinary project aiming at reconstructing the sound or timbre of ancient instruments (not existing anymore) using archaeological data as fragments from excavations, written descriptions, pictures.

The technique used is the physical modeling synthesis, a complex digital audio rendering technique which allows modeling the time-domain physics of the instrument.

In other words the basic idea is to recreate a model of the musical instrument and produce the sound by simulating its behavior as a mechanical system. The application would produce one or more sounds corresponding to different configurations of the instrument (i.e. the different notes).

Lost Sounds Orchestra
www.lostsoundsorchestra.org

The Lost Sound Orchestra is the ASTRA project orchestra. It is a unique orchestra made by reconstructed ancient instrument coming from the ASTRA research activities. It is the first ensemble in the world composed of only reconstructed instruments of the past. Listening to it is like jumping into the past, in a sound world completely new to our ears.

Since I haven’t had occasion to mention either GÉANT or DANTE previously, here’s more about those organizations and some acknowledgements from the news release,

About GÉANT

GÉANT is the pan-European research and education network that interconnects Europe’s National Research and Education Networks (NRENs). Together we connect over 50 million users at 10,000 institutions across Europe, supporting research in areas such as energy, the environment, space and medicine.

Operating at speeds of up to 500Gbps and reaching over 100 national networks worldwide, GÉANT remains the largest and most advanced research and education network in the world.

Co-funded by the European Commission under the EU’s 7th Research and Development Framework Programme, GÉANT is a flagship e-Infrastructure key to achieving the European Research Area – a seamless and open European space for online research – and assuring world-leading connectivity between Europe and the rest of the world in support of global research collaborations.

The network and associated services comprise the GÉANT (GN3plus) project, a collaborative effort comprising 41 project partners: 38 European NRENs, DANTE, TERENA and NORDUnet (representing the 5 Nordic countries). GÉANT is operated by DANTE on behalf of Europe’s NRENs.

About DANTE

DANTE (Delivery of Advanced Network Technology to Europe) is a non-profit organisation established in 1993 that plans, builds and operates large scale, advanced networks for research and education. On behalf of Europe’s National Research and Education Networks (NRENs), DANTE has built and operates GÉANT, a flagship e-Infrastructure key to achieving the European Research Area.

Working in cooperation with the European Commission and in close partnership with Europe’s NRENs and international networking partners, DANTE remains fundamental to the success of global research collaboration.

DANTE manages research and education (R&E) networking projects serving Europe (GÉANT), the Mediterranean (EUMEDCONNECT), Sub-Saharan Africa (AfricaConnect), Central Asia (CAREN) regions and coordinates Europe-China collaboration (ORIENTplus). DANTE also supports R&E networking organisations in Latin America (RedCLARA), Caribbean (CKLN) and Asia-Pacific (TEIN*CC). For more information, visit www.dante.net

Acknowledgements
NASA National Space Science Data Center and the John Hopkins University Voyager LEPC experiment.
Sonification credits
Mariapaola Sorrentino and Giuseppe La Rocca.

I hope one of these days I’ll have a chance to ask a data visualization expert  whether they think it’s possible to represent multiple dimensions visually and whether or not some types of data are better represented by sound.

* ‘described’ replaced by ‘discussed’ to avoid repetition, Feb. 10, 2014. (Sometimes I’m miffed by my own writing.)