Participants 2022

Our vision was to develop and fine-tune AI tools as “creative partners” in music-making. We brainstormed about where in the song creation process we could use AI, and it became clear AI’s capabilities were endless. In the spirit of the “AI Song Contest”, we decided to use as much AI-generated output as possible, from lyrics to melodies to beats. Meanwhile, the humble humanoid was responsible for music rearrangement and production.

For lyrics, we fine-tuned an AI model, Generative Pre-trained Transformer 2 (GPT-2). GPT-2 generates text based on input. We fine-tuned it to generate lyrics for any given combination of artist, song name, music genre and decade. 

For music, we adapted AI models from Magenta, an open source research project focused on machine learning in music and other creative work:

Magenta’s Coconet is a convolutional neural network which harmonizes melodies. Given one note, it can generate three other notes that sound good together when played simultaneously. Coconet was originally trained on Johann Sebastian Bach’s music. We fine-tuned it to harmonize the melody in a more contemporary way.

Magenta’s MusicVAE uses Variational Auto-Encoders (VAE) to generate 16 bars (30-45 seconds) of music with 3 tracks - melody, bass and drums. We used MusicVAE to generate several 16-bar samples, from which we cherry-picked nice segments.

To nicely blend all the AI output into an earworm, we used Digital Audio Workstations (DAWs) like Ableton Live and Garageband. We could adjust notes, correct pitches and also dazzle things up with Ableton plug-ins like Magenta’s Drumify.