In dialogue with a learning machine: an exploration of music, interactive theatre and live performance through Artificial Intelligence
Lure is an artistic research project related to machine learning in theatre and music. We have explored a variety of machine learning techniques in combination with musical instruments, voice and narration. The artistic research transfers questions on the topics of human–machine interaction and artificial intelligence to the fields of theatre and music.
The narrative is based on the classic novel Parzival by Wolfram von Eschenbach. Parzival meets three elegant knights, decides to seek King Arthur, and continues a spiritual and physical search for the Grail, looking for the world outside the forest in which he was raised. In Lure we have used the character of Parzival, who is now developed inside a computer, with a similar intentionality. It wants to learn about you and the world and starts a conversation with you. It tries to express itself by playing with a musician. And it leaves its virtual confinement to explore the world by itself.
With our three prototypes we studied three different artistic and technological approaches. Develop empathy with the computational character by discovering how it learns. Experience its inner workings through theatrical expression. And be touched by its musical sensitivity.
We developed an open studio event where we invited an audience for a playful session with music, performance and collective discovery. The audience was asked to interact with our character Parzival. It was an interactive presentation of our research and the three prototypes as artistic steps and even though it is work-in-progress, it was presented in a theatrically staged online performance.
Core people involved in the project are Munich composer and theatre maker Mathis Nitschke, Amsterdam experience artists Luciano Pinna and Klasien van de Zandschulp, scientists from the Munich Volkswagen Group Machine-Learning Research Lab (argmax.ai) Patrick van der Smagt, Djalel Benbouzid and Nutan Chen and Munich dramaturg Elsa Büsing.
Full Performance Video
Originally meant to happen in the physical space of Munich Pathos theatre, this project became also a prototype for a theatrical staging in digital space, due to Corona. As an audience member, imagine yourself being one of the participants in this Zoom-like meeting. You enter the space with your webcam using the same custom built Augmented Reality filter which you see on all the faces in the video.
We invite you to darken your room to get the full experience.
Chapters
00:35 PARZIVAL runs through the forest
04:10 PROTOTYPE 1: Singing with the Birds.
Parzival in musical interaction with a singer. An artistic idea of a musical dialogue between machine and performer. The user interface shows a learning process that could happen between performer and musician.
09:43 PARZIVAL and Herzeloide
12:43 PROTOTYPE 2: Jamming with the Knights.
Parzival, the machine, in playful interaction with a drummer. Simultaneously making visibly understandable what happens inside the machine.
21:00 PARZIVAL meets the knights
22:50 PROTOTYPE 3: A Chat with you, Angels!
Parzival is curious to chat with you at lure.bsqd.me
30:33 PROTOTYPE 4: What does it look like where the forest ends?
Parzival uses your answers to collect a corpus of images which it uses to make itself a picture of the world. Parzival leaves the space.
Q&A Session
directly after the Lure Open Studio Performance on 13 November 2020. With (left to right, top to bottom) Mathis Nitschke, Elsa Büsing, Luciano Pinna, Katharina Dobner, Martina Koppelstetter, Nutan Chen, Klasien van de Zandschulp, Patrick van der Smagt, Mara Widmann, Matthias Lachenmayer and Djalel Benbouzid
Review
Rita Argauer wrote a wonderful review in Süddeutsche Zeitung: “the brief moment when computer and human being came together artistically is full of horror and beauty, ambivalence and theatricality. To create something like this through a miserable computer screen is a great achievement.”
Invitation text
A curious and creative character awakens. Developed inside a computer it yearns to learn. You meet it in our performance. It wants to learn about you and starts a conversation. It tries to express itself by playing with a musician. And it leaves that space to explore the world by itself.
In our open studio we invite you to a playful session with music, performance and collective discovery. You’ll be able to interact with our character Parzival, a machine-learning model created after the mediaeval adventurer.
With our prototypes we study three different artistic and technological approaches. Develop empathy with the computational character by discovering how it learns. Experience its inner workings through theatrical expression. And be touched by its musical sensitivity.
The artistic research project “Lure” transfers questions on the topics of human–machine interaction and artificial intelligence to the fields of theatre and music. We will open our studio and show our first technical prototypes and artistic steps as work-in-progress, in a theatrically staged online meeting.
Credits
“Lure” is a project of Munich composer and theatre maker Mathis Nitschke, Amsterdam interactive artists Luciano Pinna and Klasien van de Zandschulp, scientists from the Munich Volkswagen Group Machine-Learning Research Lab (argmax.ai) Patrick van der Smagt, Djalel Benbouzid, Nutan Chen and Munich dramaturg Elsa Büsing.
with
Mara Widmann: Narrator
Martina Koppelstetter: Singer
Matthias Lachenmayer: Drummer
Katharina Dobner: Mask and fire design
Dominik Theobald: Digital Tree
Musical contributions by gamut inc, Jörg Hüttner, Martina Koppelstetter, Matthias Lachenmayer, Mathis Nitschke, Alexander Schubert
Parzival’s learning machine is based on the flat latent-variable models explained in https://argmax.ai/blog/metric/.
Besides our custom development tools we use these projects: Google Magenta, Tensorflow Pix2pix, OpenAI GPT-2, RunwayML, AIVA, Botsquad, Lens Studio, Av3ry
“Lure” is funded and supported by the Dutch Creative Industries Fund NL, the German Fonds Darstellende Künste, and the Volkswagen Group Machine Learning Research Lab.
Thanks to Pathos Theater, Mario Klingemann, Memo Akten, Vladimir Viro, Arjan Scherpenisse, Botond Cseke, Milo Schagen, Michael Schwarzfischer, Markus Krause, Tankred Dorst and Wolfram von Eschenbach.
