By Sean Zhang
Imagine being able create a continuous stream of music by merely wearing a sensor on your finger. Now, imagine being able to change that music in real-time by simply adjusting your mood.
This is ‘Biomusic’, an interface that produces music from the physiological signals from our bodies. Biomusic is a premier example of how art and science can collaborate to create innovative technologies.
The idea of Biomusic was originally conceptualized by Dr. Stefanie Blain-Moraes while she was volunteering at the complex continuing care unit for disabled children. Many of the children were completely reliant on technology and required 24-hour personal care. They had little to no ability to be to move or communicate voluntarily because, for many of them, their somatic nervous system was dysfunctional due to either injury or neurodegenerative disorders. It was tough to understand the needs and wants of these pediatric patients, as even the simple motion of pushing a button or flipping a switch proved difficult.
However, Dr. Blain-Moraes wondered if one could understand these children through the activity of their autonomic nervous system, which regulates unconscious functions, such as heartbeat, breathing rate, and sweating. Even though these children could not voluntarily communicate, one might still gain insight into their emotional world through these physiological indicators. Stefanie was eventually able to show that non-communicative children felt stimulated from being in contact with other people through changes in breathing and heart rate. However, it was tough to translate the graphs and charts she used as evidence for this effect into a meaningful interpretation for caregivers and parents who were not scientifically trained. So, she sought a way to translate these physiological signals into a medium that was relatable. As Stefanie is a classically trained pianist in addition to her science and engineering background, she wondered if this sort of scientific data could be communicated through an artistic medium instead. This was the start of the Biomusic project.
How Biomusic Works
Four main bodily signals from the autonomic nervous system are considered by the Biomusic interface: Heart rate, Skin temperature, Breathing Rate, and Electrodermal Activity (sweating) - all of which can be detected by wearing a device on one’s finger. The device plays a default tone, which then fluctuates, and changes based on changes in one’s autonomic nervous system activity. For example, breathing and heartrate are usually correlated in a relaxed individual – when you inhale, your heartrate accelerates and when you exhale, your heartrate decelerates. However, when excited, the two signals start decoupling, and the correlation is disrupted. This can be recognized by the interface and is then reflected in an accelerando in the music. In its early stages, Biomusic simply mapped changes in physiological signals to typical music tropes: Heartrate controlled the tempo of the music; Skin temperature affected the key; Spikes in electrodermal activity created little embellishments in the melody.
The iteration of Biomusic currently in development is more advanced. It is now customizable, allowing users to choose their own soundscapes rather than sticking with the default classical music one. Sounds of nature, Indian drums, or even a purring cat can be worked into the program’s musical output. Additionally, machine learning techniques are being integrated to enable the interface to recognize specific emotional states of an individual through their physiological signals and adjust the music accordingly.
A demonstration of Biomusic at its early stages. Note the dramatic change in music
(at 13:43) after the researcher asks the subject to think of something stressful!
Source: TEDxMontreal (2010)
A Plethora of Applications with Biomusic
Biomusic was initially employed to interpret the emotional states of non-communicative children. The interface was linked to children in complex continuing care units during time with their caregivers. The use of Biomusic drastically changed the interactions between caregivers and patients. Caregivers felt that even though the patients were technically non-communicative - and some even vegetative, the changes in their Biomusic let the caregivers know that the children were in fact responding emotionally during the interactions. Biomusic became a tool to communicate these emotions, giving the speechless a voice.
One other application of Biomusic is in individuals with autism-spectrum disorder as a way of regulating one’s emotional state. Stefanie teamed up with Spectrum Productions, a non-profit organization that does audiovisual design and exclusively employs individuals on the autism spectrum, to create workshops where autistic individuals could try Biomusic. In one workshop, participants went to art galleries while wearing their Biomusic interfaces. At one point during the visit, a participant noticed that their Biomusic was going haywire and wondered why. He realized that he was on the threshold between two different rooms, and that this abundance of sensory input made him feel overwhelmed. So, he stepped back into one room, and the music calmed down. Thus, Biomusic was able to make it easier for individuals on the autism spectrum to self-regulate their emotions.
In addition to being a tool for therapeutic means, Biomusic can also be used for other exciting avenues. For meditation purposes, an individual can listen to their Biomusic as a means of monitoring their heartrate and breathing. Athletes can also use the interface to gauge their performance. Even musicians and composers can use Biomusic by adding samples of their own Biomusic to their compositions. One would not only be writing music, but could actually intrinsically create different types of music by just by varying emotional states.
Biomusic is a project born from the collaboration of both science and art. While Dr. Stefanie Blain-Moraes was formally trained in science and engineering, her background in classical music lent her creative ideas to tackle problems in science. Biomusic has a promising future as both a therapeutic and creative device for patients and the general public.
Stefanie Blain-Moraes holds a B.A. Sc and PhD from the University of Toronto in biomedical engineering and rehabilitation sciences, and is an Assistant Professor in the Department of Physical and Occupational Therapy at McGill University. She has also completed her ARCT diploma in classical piano from the Royal Conservatory of Music. Her favorite composer is Beethoven. You can follow more of her work here:
http://www.moraeslab.com/biapt/
The Sci-Art Art-Sci conference “The Black Box – Biomusic by Stefanie Blain-Moraes” was made possible by the Convergence Initiative, a non-profit organization dedicated to science outreach and fostering collaboration between scientific and artistic disciplines.
This article has appeared in the McGill Daily.
Editing: Nicole Avakyan