Music, Science and Healing Intersect in an A.I. Opera

By Frank Rose 

Published Oct. 28, 2022 Updated Oct. 31, 2022 

“This is what your brain was doing!” a Lincoln Center staffer said to Shanta Thake, the performing arts complex’s artistic director, while swiping through some freshly taken photos. 

It was the end of a recent rehearsal at Alice Tully Hall for “Song of the Ambassadors,” a work-in-progress that fuses elements of traditional opera with artificial intelligence and neuroscience, and the photos did appear to show Thake’s brain doing something remarkable: generating images of flowers. Bright, colorful, fantastical flowers of no known species or genus, morphing continuously in size, color and shape, as if botany and fluid dynamics had somehow merged. 

“Song of the Ambassadors,” which was presented to the public at Tully on Tuesday evening, was created by K Allado-McDowell, who leads the Artists and Machine Intelligence initiative at Google, with the A.I. program GPT-3; the composer Derrick Skye, who integrates electronics and non-Western motifs into his work; and the data artist Refik Anadol, who contributed A.I.-generated visualizations. There were three singers — “ambassadors” to the sun, space and life — as well as a percussionist, a violinist and a flute player. Thake, sitting silently to one side of the stage with a simple, inexpensive EEG monitor on her head, was the “brainist,” feeding brain waves into Anadol’s A.I. algorithm to generate the otherworldly patterns. 

“I’m using my brain as a prop,” she said in an interview. 

Just to the side of the stage, level with the musicians, sat a pair of neuroscientists, Ying Choon Wu and Alex Khalil, who had been monitoring the brain waves of two audience volunteers sitting nearby, with their heads encased in research-grade headsets from a company called Cognionics. 

Wu, a scientist at the University of California, San Diego, investigates the effects of works of art on the brain; in another study, she’s observing the brain waves of people viewing paintings at the San Diego Museum of Art. Khalil, a former U.C. San Diego researcher who now teaches ethnomusicology at University College Cork in Ireland, focuses on how music gets people to synchronize their behavior. Both aim to integrate art and science.

... 

Wu and Khalil, the neuroscientists involved with the production, have yet to analyze their data. But at a panel discussion preceding Tuesday’s performance — and yes, this opera did come with a panel discussion — Khalil made a prediction that left the audience cheering. 

“We’ve started to understand that cognition — that is, the working of the mind — exists far outside our head,” he said. “We used to imagine that the brain is a processor and that cognition happened there. But actually, we think our minds extend throughout our bodies and beyond our bodies into the world.” 

With music, he continued, these extended minds can lock onto rhythms, and through the rhythms onto other minds, and then onto yet more. As for the spaces where that happens, Khalil said, “You can start to think of them as healing places.”