A College of Charleston research team has developed Monterey Mirror, a new interactive music performance system with artificial intelligence capabilities. The Monterey Mirror is an electronic music generator, powered by computer programming, which mirrors a human performer and can participate as an equal in a live performance. Like all mirrors, it reflects back aspects of the performer, enabling the performer to objectively hear what others hear. It is different from a recording, in that it does not repeat musical material verbatim, but instead captures deeper patterns in a musician’s style and makes them apparent. Monterey Mirror has been developed by computer science professor Bill Manaris and students with funding from the National Science Foundation. Watch a video about the project.
This spring, world-renowned composer and College of Charleston music professor Yiorgos Vassilandonakis used Monterey Mirror to compose a new piece for a mixed “ensemble” that consists of two human performers and two Monterey Mirror systems (one per performer). The Monterey Mirrors learn from the human performers and play back aesthetically similar musical variations.
“Musical performances are like a ‘game’ for musicians and Monterey Mirror elevates that ‘game’ to a higher level with more options, more control, and more fun,” Vassilandonakis says. “The first composition with Monterey Mirror 2X2, explores different modes of interaction between human performer and artificial intelligence. It was composed traditionally, like a Piano Sonata, and the performers operate as if they’re playing chamber music. What is new is their dynamic relationship with a system that can intelligently listen to what they do, and return stylistically appropriate responses that are different, but similar to the performers’ nuances.”
The Monterey Mirror project is a practical example of computing and the arts working together to inform each other and grow in tandem. The cutting-edge computing technology is readily available, and portable, which makes collaborations like this both practical and exciting.
“Being at a liberal arts and sciences institution, a place that encourages interdisciplinary exploration provided the necessary support environment for Monterey Mirror to be created,” Manaris says. “One Saturday afternoon last July, the Monterey Mirror system came to life and on my guitar, I explored with it various musical ideas for the first time. It was an amazing moment. Since then the system has evolved tremendously.”
Monterey Mirror is based on Markov models, genetic algorithms, and power-law metrics for music information retrieval. The techniques are at the forefront of computer science. Monterey Mirror trains a Markov model on a human musician’s material; then it uses a genetic algorithm, guided by power-law metrics, to discover musical “responses” that are aesthetically similar to the musician’s style. Since Monterey Mirror can work with recorded material, it is even possible to generate material in the style of musicians long-gone, such as Miles Davis and Johann Sebastian Bach.
Vassilandonakis and Manaris along with Dana Hughes, a graduate student in the computer science department, just returned from presenting Monterey Mirror at the 2011 Congress on Evolutionary Computation. The Congress is one of the leading international events in the area of evolutionary computation.
“Through this project, I’ve had the opportunity to interact and work with many brilliant people in the field,” Hughes says. “Working with both Dr. Manaris and Dr. Vassilandonakis has given two unique perspectives on what the system should do and how it should do it. Having these perspectives has been a great way to ensure that the final product performs in the best way possible. The blending of these two fields has demonstrated that software can be used as a tool for creative exploration. Not only is it amazing to realize that software is capable of being creative, but knowing how creative software works is also very exciting.”
A dedicated educator, Dr. Vassilandonakis has taught Composition and Music Theory at the University of California, Berkeley and the University of Virginia, as well as electronic music at the Centre de Création Musicale, Iannis Xenakis, in Paris, before joining the faculty at the College of Charleston in 2010. In addition to his chamber, vocal, orchestral performances and compositions, he has worked on films in the Hollywood indie movie scene. His credits include composer, conductor, and producer of scores for theater, independent films, television documentaries and commercials, as well as a theme park ride at Universal Studios, Hollywood.
Dr. Bill Manaris, computer science professor, does research on artificial intelligence, human computer interaction, and computing in music and art. Along with students, Manaris created Armonique and Armonique Lite, music retrieval systems focusing on computational aesthetics (similar to Pandora). Earlier research includes SUITEKeys speech Interface, a continuous speech understanding system for motor-impaired users. For the last decade, he has been exploring fractals, power laws, and the golden ratio and their relationship to music, art, and human aesthetics. His work has been funded by various NSF grants.
For more information, contact Bill Manaris at manarisb@cofc.edu or Yiogros Vassilandonakis at vassilandonakisy@cofc.edu.
