Participants 2022
Expecto Patronum
AI Song Contest 2022 / participantsTEAM / Expecto Patronum
SONG / 行路难 (What a hard journey)
TEAM MEMBERS / Yinan Zhou, Yifan Xie, Jiafeng Chen, Yunhe Li, Xuanqi Liu, Ziao Li
About the TEAM
We are a group of Chinese students with a shared interest in Music and Artificial Intelligence. Three of the team members, Yinan, Yifan, and Jiafeng, have a technical background in Computer Science and Engineering, whereas Yunhe, Zuyu, and Xuanqi have a musical background. We are currently based in North America and China. This contest is the reason why we met each other, and the first time we compose music together.
About the SONG
Li Bai was a famous romantic poet in the Tang Dynasty. The poem, What a Hard Journey, depicts various sceneries and the rises and falls of the poet’s emotion. It begins with confusion and depression in the journey of life, but finally, the poet shows his strong will and confidence in achieving his life goal. This poem is one of the three and is full of ups and downs in meaning, emotion, and thinking. The high spirits and unique artistic charm have made them an ancient masterpiece widely praised by later generations. It is worth noting that AI was able to capture the ups and downs in the poem.
About the HUMAN-AI PROCESS
Our idea is to train a model to learn the correspondence between ancient Chinese poetry and melody, and use this model to restore the melody of the poetry from an earlier time whose melody has been lost. To map poems to melody, we used the TeleMelody model by Microsoft (https://arxiv.org/pdf/2109.09617.pdf). Due to the lack of parallel data between lyrics and melody, this model contains two stages: the lyrics-to-template stage and the template-to-melody stage. We first built a dataset using ancient Chinese songs, whose lyrics are poetries from more recent times. Then, we trained the template-to-melody model using the built dataset.
The music materials generated by AI intervention were kept as much as possible in this piece. The major change was to make them suitable for Chinese pronunciation. This poem depicts the poet’s depression in the journey of life and shows his strong will of achieving his life goal. Therefore, the melody needs to be beautiful and vivid, along with changing harmony, giving this piece a dramatic feeling.
The general outline of the melody and harmony of each phrase uses the pentatonic scale, six-tone scale, and two parallel pentatonic scales in different tones. Then, the creation of musical texture was tuned. Finally, the rhythm and meter had been enriched or modified. It is worth mentioning that the fourth phrase has a rhythm change, resulting in a strengthening driving force in the music. The last phrase reached the climax and ended in a bright harmonic colour.