Participants 2022
AIM3
AI Song Contest 2022 / participantsTEAM / AIM3
SONG / BE WITH YOU (与你同在)
TEAM MEMBERS / Jiatong Shi (Producer & AI researcher, Carnegie Mellon University), Hang Yin (Music arranger & Songwriter, Xi’an Lingxin Culture Media), Tao Qian (AI engineer, Renmin University of China & ByteDance AI Lab SA), Keyi Zhang (Songwriter, Zhonghe Capital), Yuning Wu (AI engineer and imaginary designer, Renmin University of China), Shuai Guo (AI engineer, Renmin University of China), Zhaodong Yao (Musical advisor, Tottori University), Huazhe Li (Musical advisor, Tsinghua University), Peter Wu (Technical advisor, University of California, Berkeley), Qin Jin (General advisor, Renmin University of China)
About the TEAM
The AIM3 team is a joint team with both musical specialists and AI researchers. The passion for music and computer algorithms brought us together across the Pacific Ocean from various backgrounds. Our team members are primarily from academic institutions. Some are also from the musical, technical or financial industries.
About the SONG
Our song creation stems from our thinking about the relationship between humans and artificial intelligence (AI), which was originally designed to support humans by providing assistance with simple problems. However, with the continuous development, we expect that AI will eventually be able to deeply understand human beings and share pleasure with human beings. We imagine this future in our song, which is born through a combination of various deep learning algorithms and in-depth collaboration with humans.
The highlights are as follows:
We explore extensive collaborations between humans and AI in three aspects: melody generation, lyrics generation, and singing voice synthesis.
We apply AI algorithms to push our performance limits, which we cannot achieve alone.
We emphasize the diverse perspective of different groups within both the creation and performance of our song.
About the HUMAN-AI PROCESS
Our song creation process includes three general stages: Lyric & Theme Generation, Music Score Preparation, and Music Production. The detailed workflow is illustrated in the figure on the blog. All stages combine the effort from machine learning models (in orange blocks), music specialists (in yellow blocks), and the general public (in blue blocks).
We adopt three models for lyric generation, theme/melody generation, and singing voice synthesis, respectively. The lyrics generation model applies a modified architecture derived from T5 language model and SongNet lyrics model; the theme/melody generation model uses TeleMelody for conditional melody generation; the singing voice synthesis model applies the ACE Singer and Bytesing V3. (Please refer more technical details in our blog)
The final song is a joint effort from
AI models: lyric generation model, theme/melody generation model, and singing voice synthesis model.
Music specialists: lyric & melody refinement, music arrangement, digital music production (including recordings), and music mixing.
The general public: voting over lyrics, lyric-theme/melody pairs, and polish of singing voice synthesis model with pitch contour modification.
Lyrics
Singer
世界曾寂静得一如既往(The world was as silent as ever)
直到你在我身旁 (Until you are by my side)
生命便倔强得无人可挡 (My life is stubborn and unstoppable)
你就是我的翅膀 (Since you are my wings)
带我穿过迷惘 (Take me through confusion)
未知染上了渴望 (The unknown is sparked by our desire)
释放我想象 (Release my imagination)
未来不同寻常 (The future is extraordinary)
带我告别惆怅 (Take me goodbye to melancholy)
时间洒满了希望 (Time is full of hope)
世界的万象 (Everything in the world)
咫尺可及的力量 (The magic to touch and feel the world)
就在我手掌 (Is now in our hands)
-----
世界曾寂静得一如既往(The world was as silent as ever)
直到你在我身旁 (Until you are by my side)
生命便倔强得无人可挡 (My life is stubborn and unstoppable)
你就是我的翅膀 (Since you are my wings)
带我穿过迷惘 (Take me through confusion)
未知染上了渴望(The unknown is sparked by our desire)
释放我想象 (Release my imagination)
未来不同寻常 (The future is extraordinary)
带我告别惆怅 (Take me goodbye to melancholy)
时间洒满了希望 (Time is full of hope)
世界的万象 (Everything of the world)
咫尺可及的力量 (The magic to touch and feel the world)
就在我手掌 (Is now in our hands)
去探索 (To explore)
回忆也会更深刻 (Memories will be deeper)
怀念的梦想的 (Missing and dreaming)
每一天都能叫醒耳朵 (Wake up your ears every day)
啦啦啦啦 (Lalalala)
你还想去体会什么 (What else do you want to experience)
带我穿过迷惘 (Take me through confusion)
未知染上了渴望 (The unknown is sparked by our desire)
释放我想象 (Release my imagination)
未来不同寻常 (The future is extraordinary)
带我告别惆怅 (Take me goodbye to melancholy)
时间洒满了希望 (Time is full of hope)
世界的万象 (Everything of the world)
咫尺可及的力量 (The magic to touch and feel the world)
就在我手掌 (Is now in our hands)
AI
你带我去远方 (You take me far away)
陪我穿越时空 哦 (Through time and space, oh)
我听见你在唱 (I hear you singing)
这未来如你所想 (The future will turn out as you wish)
带我穿过迷惘 (Take me through confusion)
未知染上了渴望 (The unknown is sparked by our desire)
释放我想象 (Release my imagination)
未来不同寻常 (The future is extraordinary)
带我告别惆怅 (Take me goodbye to melancholy)
时间洒满了希望 (Time is full of hope)
世界的万象 (Everything of the world)
咫尺可及的力量 (The magic to touch and feel the world)
就在我手掌 (Is now in our hands)
-----
过迷惘 (Through confusion)
未知染上了渴望 (The unknown is sparked by our desire)
我想象 (My imagination)
未来并不同寻常 (The future is extraordinary)
啦 (La)
啦啦 (Lala)
啦 (La)
啦 (La)
啦 (La)
跟着我跟着我 (Follow me, follow me)
去浩瀚世界去探索 (To explore the vast world)
听我说听我说 (Listen to me, listen to me)
回忆也会更深刻 (Memories will be deeper)
怀念的梦想的 (Missing and dreaming)
每一天都能叫醒耳朵 (Wake up your ears every day)
告诉我告诉我 (Tell me tell me)
你还想去体会什么 (What else do you want to experience)
过迷惘 (Through confusion)
未知染上了渴望 (The unknown is sparked by our desire)
释放我想象 (Release my imagination)
未来不同寻常 (The future is extraordinary)
带我告别惆怅 (Take me goodbye to melancholy)
时间洒满了美好希望 (Time is full of beautiful hope)
世界的万象 (Everything of the world)
咫尺可及的力量 (The magic to touch and feel the world)