ABOUT THE TEAM

Our team consists of a lot of student power with a pinch of doctor. During the Sound and Music Technology Course given by Dr. Anja Volk at the Utrecht University, five students teamed up and found their enthusiasm for human-AI co-creation which led to dedicate their course project to this very topic. With results quite worth listening to, together we have finally decided to team-up for this year's AI Song Contest with a composer student from ArtEZ University of the Arts. Our team composition equips us with solid machine-learning- and a fair amount of music production skills. Moreover, our team embodies diversity with people from around the world: The Netherlands, Russia, China, and Germany. 

ABOUT THE SONG

We created an AI-driven song to utter the unreflected musicality in which we can be both listeners and performers inside our minds. We see this concept particularly important nowadays when shared musical experience is impossible and everything is happening in the solitude of our rooms. 

This is why we introduce The Matrix reference and find both melancholy and inspiration in it. We concentrate not on the dystopian idea of overpowering AI, but rather on us, both as producers and listeners, being isolated in cells of our workspaces and locked inside our heads. 

In this sense, we are not so different from AI as people see it, and the ideas we intended to put into AI’s mouth turn out to be applicable to ourselves. If so, can we really expect an artificial mind to solve our problems? Can it not suffer like we do while experiencing what we do?

ABOUT THE HUMAN-AI CO-CREATION PROCESS

The creation of the song can be seen as a dialogue between machine learning people and producers. Two different models were used: MusicVAE for generating melodies and PolyphonyRNN for chords. The choice was made since the former delivers monophonic and the latter polyphonic output. Instead of extracting specific genre characteristics, we aimed at combining several genres and techniques into an experimental setup that can imbibe and impersonate musical interests of the whole team, including ambient, swing jazz, game music, EDM, and more. 

We selected four generated MIDIs with catchy and recognizable patterns to use for melodies throughout the entire song. One of the MIDIs contained an intro melody, upon which we based the chord progression, so it was a sort of starting seed for the creative process. After the melody and chords were present, drums were added. 

We augmented our song with the voice in a bipartite pipeline. We first generated the lyrics by feeding The Matrix script to GPT-2 and then used this model’s output as an input for speech synthesis. The latter was performed using a framework based on the Tacotron 2 model. To support the feeling of sincerity and collaboration in our song, we have used our own voices as a base for synthesizing the speech.