Skip to main content

New York Times reports on CACSSS Researcher Dr Alexander Khalil’s (School of Film, Music & Theatre) collaborative music and neuroscience work

Dr Alexander Khalil (UCC) and Dr Ying Choon Wu (UCSD) have been working with Google AI specialist K Allado-McDowell, composer Derrick Skye, and data artist Refik Anadol on a project that is both a performance of an opera, some elements of which are generated using AI; and an experiment designed to measure audience response to various moments in the opera. Khalil and Wu collect behavioural data from a large number of audience members, resulting in a type of “heatmap”, indicating points of audience engagement in the performance. This heatmap then guides analysis of brainwave and interview data recorded with a relatively small number of audience members.

A subscription is required for the full article but see the excerpt below:


Music, Science and Healing Intersect in an A.I. Opera

By Frank Rose 

Published Oct. 28, 2022 Updated Oct. 31, 2022 

“This is what your brain was doing!” a Lincoln Center staffer said to Shanta Thake, the performing arts complex’s artistic director, while swiping through some freshly taken photos. 

It was the end of a recent rehearsal at Alice Tully Hall for “Song of the Ambassadors,” a work-in-progress that fuses elements of traditional opera with artificial intelligence and neuroscience, and the photos did appear to show Thake’s brain doing something remarkable: generating images of flowers. Bright, colorful, fantastical flowers of no known species or genus, morphing continuously in size, color and shape, as if botany and fluid dynamics had somehow merged. 

“Song of the Ambassadors,” which was presented to the public at Tully on Tuesday evening, was created by K Allado-McDowell, who leads the Artists and Machine Intelligence initiative at Google, with the A.I. program GPT-3; the composer Derrick Skye, who integrates electronics and non-Western motifs into his work; and the data artist Refik Anadol, who contributed A.I.-generated visualizations. There were three singers — “ambassadors” to the sun, space and life — as well as a percussionist, a violinist and a flute player. Thake, sitting silently to one side of the stage with a simple, inexpensive EEG monitor on her head, was the “brainist,” feeding brain waves into Anadol’s A.I. algorithm to generate the otherworldly patterns. 

“I’m using my brain as a prop,” she said in an interview. 

Just to the side of the stage, level with the musicians, sat a pair of neuroscientists, Ying Choon Wu and Alex Khalil, who had been monitoring the brain waves of two audience volunteers sitting nearby, with their heads encased in research-grade headsets from a company called Cognionics. 

Wu, a scientist at the University of California, San Diego, investigates the effects of works of art on the brain; in another study, she’s observing the brain waves of people viewing paintings at the San Diego Museum of Art. Khalil, a former U.C. San Diego researcher who now teaches ethnomusicology at University College Cork in Ireland, focuses on how music gets people to synchronize their behavior. Both aim to integrate art and science.


Wu and Khalil, the neuroscientists involved with the production, have yet to analyze their data. But at a panel discussion preceding Tuesday’s performance — and yes, this opera did come with a panel discussion — Khalil made a prediction that left the audience cheering. 

“We’ve started to understand that cognition — that is, the working of the mind — exists far outside our head,” he said. “We used to imagine that the brain is a processor and that cognition happened there. But actually, we think our minds extend throughout our bodies and beyond our bodies into the world.” 

With music, he continued, these extended minds can lock onto rhythms, and through the rhythms onto other minds, and then onto yet more. As for the spaces where that happens, Khalil said, “You can start to think of them as healing places.” 


The project described in the NY Times was a pilot, featuring a 40-minute excerpt of the larger opera, performed for an audience of about 1000 at the Lincoln Center. A full version of the project is planned for next year.


Welcoming the news Prof Griff Rollefson in the Dept of Music saying

This is a pretty exciting sneak peek at some forthcoming developments in the School of Film, Music and Theatre, its new Centre for Arts Research and Practice (CARPE) and its main interface with the Future Humanities Institute, the Arts Research & Practice Cluster

College of Arts, Celtic Studies & Social Sciences

Coláiste na nEalaíon, an Léinn Cheiltigh agus na nEolaíochtaí Sóisialta

College Office, Room G31 ,Ground Floor, Block B, O'Rahilly Building, UCC