Jury and Evaluation

We received more entries than ever this year, and so we divided our evaluation process into three steps.

Opening Round

We first assigned each entry to two different members of the organising team. These members made a preliminary assessment of each entry based on the song itself, the process document, and any other materials provided for the team page: 0 for entries with too little information to make a serious evaluation, 1 for entries that were unlikely to be competitive, 2 for entries that were possibly competitive, and 3 for entries that were definitely competitive. If either one of the preliminary assessors scored an entry with a 2 or higher, the entry continued to the semi-finals. Of the 67 teams in the opening round, 33 qualified for the semi-final.

Semi-Final Round

The 33 semi-finalists went out to a professional jury to be evaluated more carefully according to our four contest criteria:

• the song itself;

• pushing the boundaries of creative expression;

• workflow, collaboration, and the songwriting process; and

• diversity, ethical, and cultural considerations.

Each entry was evaluated independently by at least three jurors. After receiving all scores, we added or subtracted an adjustment to each juror’s scores where they were systematically higher or lower than average: this prevents any team from unfairly benefitting from or being penalised for having been assigned to jurors who tended to be more generous or more strict. The jurors also flagged potential rights issues, which the organising team reviewed before determining the list of finalists.

The 10 semi-finalists with the highest average adjusted score from the jurors continued to the final round.

Final Round

The public now has a chance to vote on the finalists. Any one person is allowed to vote only once (we use technology to mitigate the effect of ‘ballot stuffing’), but voters vote for as many finalists as they like on their ballot (not just one).

The overall winner will be determined by a 50–50 average of the jury scores and the public vote.

A Final Word

The AI Song Contest is grateful to all of the jurors who have worked with us over the years, but we wish to give a special shout-out to Mark Simos of the Berklee School of Music, who has used his experience from his Songs for Social Change contest to help us refine our evaluation techniques.