University of Sussex homepage

The University of Sussex this week saw the performance of an opera with a twist … it was sung by robots.

After weeks of intensive singing lessons by programming experts, two Nao robots took centre stage at a symposium bringing together research into music and artificial intelligence.

Dr Evelyn Ficarra, Lecturer in Music at the University of Sussex and Assistant Director of the Centre for Research in Opera and Music Theatre (CROMT), conceived of and led on the project, and said: “This has been a fascinating project, with potential for impact beyond opera and music theatre.

“If, in the near future, we are expecting to see robots used as care workers or teaching assistants, then we need to teach them to understand and respond appropriately to humans. The virtues of the musician – listening, co-operation, group creativity – are transferable skills that could apply in all kinds of human situations. Opera requires all of these, plus vocal expression, acting skills, movement and the ability to respond to other performers.

“So, in addition to being a fascinating exploration of ‘post human’ performance, the work could have interesting implications for research in artificial intelligence and social robots.”

Dr Ficarra composed one opera for two robots and cello, ‘O, One’, which is partly sung in binary code. The robots were accompanied by Dr Alice Eldridge, cellist and Research Fellow in Digital Performance. Professor Ed Hughes, Head of Music at the University of Sussex, composed a second opera, ‘Opposite of Familiarity’, with librettist Eleanor Knight.

They collaborated closely with Dr Ron Chrisley, Reader in Philosophy and Director of the Centre for Cognitive Science (COGS), who has been programming the robots, and with Tim Hopkins, an opera director and researcher at CROMT, who staged the operas.

Dr Chrisley said: “We decided at the beginning that we would not take the easy route of recording human voices and having those sound files played by the robots. Rather we wanted the robots to ‘sing’ with their own synthesised voices.

“However, the Nao was not designed to sing: the hardware and software supplied with the Nao robots only allow for a limited variety of interventions into the robot’s speech sounds. It’s been an interesting task subverting and re-purposing the Nao’s speech engine, grafting the composers’ intentions onto pre-programmed vocal algorithms, and finding ways to express pitches and rhythms.”

Dr Ficarra continued: “The process of composing for robotic voices has been very different than for humans, and has been a really collaborative process. We have had to learn how robots vocalise and to fit the music to their physicality and internal programming. We first taught them to sing arpeggios but extending a single note, for example, was a bigger challenge and resulted in some interesting vocal effects.”

Professor Hughes said: “Having written two operas for humans before, it is really strange and quite touching to see robots apparently expressing what we think of as human themes of recognition and even tenderness through opera.”

Robot Opera, a mini symposium, took place on the University’s campus on Thursday 15 June. As well as the robot performance there were wider talks and discussions about the philosophy and potential impact of artificial intelligence and the arts.

The event was organised in conjunction with the Centre for Research in Opera and Music Theatre and the Centre for Research in the Creative and Performing Arts (CRCPA).