Music is one of the most powerful forms of communication the human race has come up with. When it comes to sending purely emotional messages of joy, despair, exhilaration, languor, or what scientists call “dynamic” emotions — like when you’re excited and terrified at the same time — few languages communicate as efficiently and effectively as music.
Psychologists and others who study the human brain know that emotions have a strong physiological component. Smiling lowers the blood pressure; anxiety constricts capillaries; anger makes your adrenal glands secrete. But when someone plays music and someone else listens to it, what changes in his or her body and how, if at all, are those changes linked? Can the transmission of emotions through music be measured and studied?
That’s the question behind a fascinating experiment that took place on a sunny Sunday in April at Boston’s Symphony Hall, with the Boston Symphony Orchestra and an audience full of children and their parents. The results of the experiment aren’t going to come quickly, but the group of musicians, psychologists and technicians in the Green Room that afternoon gathered enough data about the physiological effects of music to keep them busy for months and, just as important, to point the way toward further research.
If you’re going to measure how music affects an audience, you will need a high-fidelity transmission medium, and there are few better such media than Symphony Hall. Its acoustics are world-renowned, and everyone involved in music there — audiences, musicians and the many recording engineers, including myself, who have plied their trade there — loves to work in the place. As it happens, my first gig when I moved to the Boston area was to mix the radio broadcasts of the Boston Pops orchestra under the late Arthur Fiedler from the stage of Symphony Hall. Since then, I’ve attended many concerts, rehearsals and recording sessions, but none that required as much audience participation, and as much technology spread throughout the hall, as April’s event.
The project was dreamed up by three researchers whose fascinating résumés straddle the worlds of music and science. Teresa Marrin Nakra, who happens to be a personal friend, got an orchestral conducting degree from Harvard and then did her doctoral work at MIT’s Media Lab, where her thesis was a “conductor’s jacket”: a wired-up garment equipped with sensors to measure motion, muscle tension and other physiological factors while the wearers — Boston Pops musical director Keith Lockhart among them — conducted symphony orchestras. She’s the artistic director of a nonprofit group that develops new technologies for the performing arts called Immersion Music, and she recently became an assistant professor at The College of New Jersey.
Daniel Levitin dropped out of both MIT and Stanford as an undergraduate, and spent his 20s and 30s as an A&R man for a small record label in San Francisco, at the same time freelancing as a musician, engineer, producer, standup comedian and music journalist, counting among his clients Mel Tormé, Steely Dan, Blue Öyster Cult, Jay Leno and Mix magazine. Eventually, he went back to Stanford and got a degree in psychology and went on to receive a Ph.D. from the University of Oregon, where his thesis was on absolute (or “perfect”) musical pitch. He joined the music faculty at McGill University in Montreal in 2000, and now has a joint appointment as an associate professor in music and psychology. His book This Is Your Brain on Music: The Science of a Human Obsession will be published this summer by Dutton.
Stephen McAdams, also at McGill and also with a joint appointment, is an expert on musical perception; he has a Ph.D. in hearing and speech sciences from Stanford and a doctorate in cognitive psychology from the University of Paris.
The project originated, says Nakra, when Levitin called her up about a year and a half ago and told her he had received a grant, which he wanted to spend on some kind of collaboration with her and McAdams. “We met in Montreal and came up with this idea,” she says. “It took in a bit of all of our research: Steve had worked with slider boxes as response tools, and Dan had worked with audience response to music. So we decided to combine our methodologies and do a project that was bigger than anything each of us could do alone.
“We wrote up a proposal and then asked, ‘What orchestra might agree to do this?’ I called a friend in the BSO’s education office, and they immediately saw the value in the experiment and that it would be best to integrate into a concert with both kids and parents present. They also knew that Keith Lockhart would be the right guy to conduct.”
In fact, Lockhart was scheduled to conduct one of the orchestra’s Family Series concerts. Myran Parker-Brass, the BSO’s director of education and community programs, approached him over the summer. In the hall’s parking lot on July 4, as Lockhart was getting ready to conduct the Pops’ huge outdoor Independence Day extravaganza, he gave her the thumbs up.
What Lockhart agreed to was to be wired up with an updated version of Nakra’s jacket, made from a Lycra biking jersey. Inside the jacket were an electrocardiogram (EKG) sensor, an accelerometer and six electromyogram (EMG) sensors from a company called Delsys. These were all wired to a beltpack, and then through a 100-foot audio snake to a computer backstage in the Green Room.
Electromyograms measure muscle tension. Delsys, which makes these sensors and others for the medical and sports-training industries, has an academic origin — it grew out of a program in neuromuscular research at Boston University — which may be one reason why they were so helpful to the project. They’ve sponsored Nakra in her research in the past, but, she says, “I didn’t expect them to come through like this. They loaned us about $60,000 worth of gear. The project would have been unaffordable without their equipment. Plus, they helped us build a custom isolation unit for each of the musicians so they wouldn’t be harmed by stray electricity — like if lightning struck the hall.” What do they get out of it? “Well, they get nice pictures of Keith for their Website,” she says with a laugh. “They like the idea of having arts applications for their products, too.”
In addition to the sensors, Lockhart wore an armband made by the Pittsburgh-based company BodyMedia containing a two-axis accelerometer, a body temperature gauge and a galvanic skin response monitor, which measures a person’s excitement: As the excitement level increases, skin resistance goes down. The armband, which is the sort of thing you might see worn in very posh health clubs, wasn’t wired to anything; it collects data by itself. The data can then be downloaded via a USB port. However, the team time-synched the device with the main computer before the concert to make sure all of the data would be coordinated.
In her previous work, Nakra was able to determine that a conductor experiences physiological changes along with the music, but with this experiment, she was able to ask, “How are those signals received?” The first people to receive them would be, of course, the players in the orchestra. So five musicians were wired up similarly to the conductor, with armbands, EKGs and two EMGs each. “All of the members of the BSO got a letter explaining what we were doing,” says Nakra, “and the five we ended up working with were the first to respond. They were among the younger, more science-y types in the orchestra; definitely not the older crowd.” The players were nicely spread throughout the orchestra: one player each from the bassoon, percussion, trumpet, bass and violin sections.
And finally there was the audience. “The orchestra has a ‘Kids Club,’ who are regular subscribers to the Family Series,” explains Nakra. “We sent all of them a letter inviting them to participate. We told them we were doing an experiment in how science can reveal emotions in music. Several wrote back and signed up, but we still needed to recruit some audience members at the last minute. We had a bunch of volunteers from McGill, Immersion Music and Northeastern University greeting people in the lobby and getting them equipped.”
Sixteen audience members — children and parents — got the BodyMedia armbands, as well as finger cuffs made by Thought Technology, a Montreal biofeedback company, to measure blood pulse volume, heart rate and galvanic skin response. Four more were given wireless heart-rate monitors. In addition, 30 concertgoers were given boxes with slide potentiometers on them marked “weak” and “strong” at the two ends. They were given instructions to “Continuously rate your impressions concerning the strength of the emotional reaction you have to the music as you are listening” and move the slider accordingly. The instructions were careful to point out that their response should not be determined by whether or not they liked the music — just how strongly they responded to it, either positively or negatively.
Who got the sliders depended on where they sat: The researchers didn’t want to run the ribbon cables connecting them more than 100 feet, and so only people in the front few rows, on the side of the hall closest to the Green Room, could get them. “Through the education office, the box office sent letters to everyone who had tickets in that section,” says Nakra, “and people responded. But the worst part was getting everyone to sign releases just before the concerts, and so we had a big team of guys running around with clipboards.”
Another interesting chore was running the cables, which couldn’t commence until after 10:30 the night before the experiment as there was a concert in the hall that evening. It kept the researchers and volunteers very busy until past two in the morning. “We could get under the stage, which was a big help,” recalls Nakra. “And it was important to protect the cables. The ones connecting the beltpacks to the isolation boxes cost about $700 each; they’re very lightweight and were built for laboratory experiments, not heavy-duty use. One hit from a stiletto heel and they would have been toast.
“It was pretty strange under there,” she continues, laughing. “Piles of cables and some evidence of rodents, but the weirdest thing is that there are holes in the stage and so there are these odd streams of light coming through. Crawling around down there I sort of felt like Indiana Jones.”
The concert-cum-experiment, which was also recorded and videotaped in HD by a crew from the local PBS station, went off almost without a hitch. During the first number, a Mozart overture, one of the accelerometers fell off of Lockhart’s wrist. The researchers realized that something had gone wrong when that sensor’s data stream, which was being processed using custom software that Nakra’s husband Jahangir wrote, turned into gibberish. After the piece ended, they could see on their video monitor the sensor hanging down and flopping around, so after a brief debate, one of the technicians went out onto the stage with a roll of duct tape. “Be careful,” warned Lockhart as the man started to re-attach the sensor to the conductor’s Lycra sleeve, “it’s Armani.” It got a big laugh, at least from the parents in the audience.
So what did the researchers get out of all this? They don’t know yet. “It’s our summer project,” says Nakra. Not counting the video and audio, they gathered 4 GB of data, which they now have to sift through. “The first task is to look at the muscle activation of the performers. We expect to see the score realized in that data pretty clearly. We’re most curious about heart-rate activity. It was the first time we have collected high-resolution, clean EKG data. We’re wondering how the heart rates of the performers and the audience will correlate. Not that they’d beat in sync, but whether they’d move up and down together.
“Then we’ll look at the skin response, the excitement level. We expect that the performers’ response will be more intense than the audience’s, but we don’t know for sure. Performers are so used to this and comfortable with it that they might be more casual about it than the audience. And we’re looking at delays: The conductor’s responses will be a little ahead of the music, the performers will be in the moment, although there is some anticipation there, too, when they prepare to play a phrase, and the audience’s will be a little behind. But we want to see how large those delays really are.”
A second experiment is scheduled soon, which will play the recordings from the BSO concert to a similarly wired audience. The main question this experiment hopes to answer is how different, and in what ways, an audience’s response to a recording is from an audience present at a live event.
Nakra, Levitin and McAdams know that they’re just getting started on this idea of quantifying physiological responses to music, but their work could turn out to be very important to the way people make, hear, record and, dare I say it, sell music. Not that we need any more bean counters determining what music we listen to, but these are scientists and musicians, as well. Besides, as Lockhart said in a television interview just before the concert, “They’re measuring something that musicians intuitively have always known is there: the power of music to transport you.” It will be interesting to see what kind of science can be found driving that mode of transportation.
Paul D. Lehrman wrote a satirical article on the practical uses of perfect pitch in 1977. It was never published, but his predictions have all come true.
