Participants 2024
About the TEAM
Keon Ju Maverick Lee (he/him) is an electronic percussionist, drummer, and music composer specializing in designing musical agents for percussive musical practice as a Ph.D. candidate at Simon Fraser University's Metacreation Lab for Creative AI. He collaborates with the Centre for Digital Music in London to create a new MIDI dataset. Recently, he performed with a Media Artist on the ReVerie project at Ars Electronica Deep Space and exhibited at SIGGRAPH in 2024.
About the SONG
We created “Echoes of the Synthetic Forest” using a training corpus mainly comprised of our own compositions. As ambient producers, we focused on sonic texture and harmony, creating a soundscape that feels like you’re taking a relaxing walk through a surreal forest of robotic birds and crunchy distorted leaves. Our approach was to use a musical AI agent able to remix our musical styles with limited data as the base of the piece. We then add in our own percussion and guitar performances on top of the AI track to close the loop between the AI agent and human musicians.
About the HUMAN-AI PROCESS
We started out by creating original music to be used as training data, which was then supplemented with other ambient music. Our training data consisted of an ambient meditation track featuring lots of sustained chordal drones, and a more rhythmic downtempo piece created with Tidal Cycles live coding and processed electric guitar. Our source compositions feature a balance of natural soundscapes and glitchy noise-based synths. We used these compositions as training data for an AI performer agent.
Our AI musical agent (MACataRT) was developed by Keon as part of his PhD thesis. MACataRT processes a single audio file of training data in real time, using audio mosaicing to segment and rearrange sounds based on timbral features. We used this AI agent to synthesize the core audio for our song, generating multiple different sections of audio that we would then stitch together in Ableton.
During the process of mixing and arranging the AI agent output, we used the HARD(HArmony-Rhythm Disentanglement) Remixer audio plugin. This is an AI DJ plugin that extracts harmonic and rhythmic features from two audio sources, and output a combined signal. We used the plugin to smoothly transition between song sections, and for layering different agent clips on top of each other; combining the harmonic components of a synth with the rhythm of snapping, bird song, and rushing water, for instance.