ABOUT THE TEAM

Rubato Lab was created in 2018, we would like to introduce the artificial intelligence piano model that was exhibited in 2019. The music AI model was made with Music GPT-2 that was modified from GPT-2. We used music transformer instead of pre-trained transformer. We would like to further develop, study, and publish kinds of AI music models.

ABOUT THE SONG

Before the sunrise, you can feel a lot of emotions at the point of change from cold to warm. We wanted to feel the atmosphere where coolness and warmth coexist with music once again. 

So, AI created a warm atmosphere with drums and basslines, and we created a melody with a cold atmosphere. Our music style is close to Lofi, but you can get a warmer feel. It's a blessing that AI can create warm music. The feeling of AI is often a cold feeling that can't be comforted when we think about it, but we've created a warm atmosphere with AI. 

The world is struggling with Covid 19. We made this music with a passion that endures with this situation. We hope this music doesn’t lose hope while feeling the feeling just before sunrise, which offsets the cold feeling with the warm feeling. 

ABOUT THE HUMAN-AI CO-CREATION PROCESS

We used Music GPT2 to create bass lines and LSTM VAE to create drum beats. The preprocessed dataset crawled MIDkar’s MIDI data and used the crawled MIDKar and Lakh data. Each dataset consists of one vocab representing note length (duration), note number, and velocity. For example, a note with a quarter note is expressed as follows. “[0.25, [63, 66, 70],100]” 

In the same way as above, all the notes of the time step were learned by the model and became the basis for making music. If pre-processing is performed in the above method, data can be further reduced than the piano-roll method, and data can be arranged according to a one-time step or the length of the note. 

We tried to train the model in the way that the existing NLP model learns, and applied the meaning of one-note as a single word. Unlike the existing approach, the problem of increasing the size of the vocab a lot occurred, so 50000 notes that appear frequently were selected and used as vocabs, and other notes or codes were processed with UNK tokens. 

The chords and melody of the sound source were composed by four team members based on the model's baseline, and vinyl noise was added to bring out the sensibility of the Lo-fi hip hop music, and finally, the song was completed through mixing.