Case Stories / A brand new dance partner: Teaching AI to dance

A brand new dance partner: Teaching AI to dance

As data scientists gradually inch machines toward human levels of intelligence, they’ve begun to tackle some very human endeavours. At Peltarion, an artificial intelligence (AI) expert and dance choreographer teamed up to find out if an AI model can not only learn a dancer’s style, but create its very own artistically compelling choreography.

02/In context

Art has always worked hand in hand with technology to push at the forefront of what’s possible. From mining gold for jewelry to rendering 3D graphics, technology has long been used to aid in the progress of creating art and even developing new art forms. However, with AI advancements, technology isn’t just adding to a palette or reinventing a medium anymore - it’s reinventing the artist themself. Choreographer Louise Crnkovic-Friis explains, “This project raises a great deal of interesting questions in regard to what art really is and who can be an artist. These are questions which I believe are of huge importance for the art world to explore.”

03/The challenge

In 2015, Louise challenged her husband, Peltarion CEO Luka Crnkovic-Friis, to find an AI application which could be incorporated into her work and further strengthen the process of creating new choreography. Naturally, she had her doubts as to whether this process would be possible at all. “Dance is so abstract, so individual,” Crnkovic-Friis recalls. “I wasn’t sure the system would be able to get even close to creating something which could be useful for me in my work. But of course, I was very curious to find out.”

Teaching a computer to generate world-class dance choreography sounds especially difficult because it’s not easy to describe dance in the first place, even to other humans. As dependent on style and emotion as it is technique, the syntax of dance has never been clear cut for humans, so why should it be for machines?

The only way to find out was to try, and like any AI project, the work started with gathering data points. Using an inexpensive Kinect V2 sensor, the data scientists collected raw movement data from the choreographer dancing her own pieces. “My job was in producing the data,” she says. “Dancing for several hours at end in front of the Kinect sensor allowed for the choreographic material to be recorded.”

04/The results

The beauty of starting from the most basic building blocks of dance – spatial movement data – is that the AI model could create a syntax by understanding relations between the dancer’s anatomical joints. Utilizing a recurrent neural network they dubbed chor-rnn, the AI expert-dancer team fed the model 13.5 million spatiotemporal joint positions from five hours of contemporary dancing. After six hours of training the model, its choreography started to resemble human motion, all the joints moving in the proper way. After 48 hours, the model could not only choreograph new dances, but do so in a style remarkably similar to the style of the choreographer.

Crnkovic-Friis was thrilled: “It started out resembling something like Bambi on ice. But after just 48 hours, the movements started to resemble something which I thought could definitely be my own choreography. I was just amazed about these results, and the fact that AI could produce these kind of achievements in such a short amount of time.”

This revelation doesn’t mean choreographers should hang up their dancing shoes just yet, though. To the choreographer driving this model, it’s still just a tool to complement her taste and style, not replace it. “I do different collaborations with the system. I've done straight-up AI choreography, where the AI generated the choreography in full, with me teaching the choreography to a dancer. I have also gotten a first iteration of an AI-produced choreography, worked with it, making appropriate adjustments and iterations such as giving it more defined momentum and emotion, and then giving it back to the system in order to get the right choreography.”

Chor-rnn after 48 hours of training
Chor-rnn after 48 hours of training

This flexible relationship to AI means the artist can actually engage with her own work more than ever before, establishing a creative partnership with herself, where learning and improvement is a two-way street. And, of course, the AI still has plenty of room to improve. Crnkovic-Friis explains, “Where the AI is weak is on the semantic level, having a feeling for not just the individual movements and sections, but the whole piece.”

05/Conclusion

For Crnkovic-Friis, the question isn’t so much whether the AI will reach that next level of artistry, but rather how artists will choose to adopt such an interesting tool into their work. And of course, not every choreographer can play around with AI the way she has been able to – being married to an AI expert and all. However, the beauty of the Peltarion Platform is that it has the ambition of eventually making AI usable and affordable for artists of any kind.

Louise Crnkovic-Friis

Crnkovic-Friis says, “We're going to reach that next level. However, we as humans have this belief that we are so complex, so how could an AI system possibly be able to learn my thoughts and my very own artistic expressions? Questions such as ‘When is something art, and when does something stop being my art?’ are prone to arise.”

Yet, the prospect of how AI can further her art outweighs her concerns: “The work of an artist is to be curious and open-minded, to investigate. For me, AI gives me something new. It takes me into another universe and allows me to create something that is true to me.” Whether all artists agree that AI should be involved in their practice of art or not, it certainly opens a frontier of new possibilities, with plenty of room to grow for those curious enough to explore.

Want to know more? Read the research paper 'Generative Choreography using Deep Learning' on arXiv.org

---

Photos by Kim Glud & Ahmad Odeh

Get started for free