Analyzing Music that Escapes Conventional Notation: Towards Automated Spectrogram Segmentation and Annotation

University of Calgary
1106-3600 Brenner Drive NW Calgary, T2L 1Y2

Since the nineteenth century, the study of music has revolved around a musical score as the central representation of a work. However, in the case of much contemporary and computer music, conventional and even extended musical notation is often insufficient. Consequently, we can only access this complex and innovative music through deliberate and active listening. Reliable visual representation of important musical parameters, including pitch, duration, intensity, and attack/decay, is required to facilitate the study and evaluation of this music. Several researchers have developed computer assisted music analysis programs to facilitate further exploration of musical materials. Three such iterations by Donin (2004), Clarke, Dufeu, Manning (2014), and Burleigh, Sallis (2011) have different philosophical approaches regarding the musical analysis as well as its presentation to the user, yet all three rely heavily on the manual annotation, and description of the musical materials or spectral components in question by the analyst.

The fundamentally different strategy of this project is the automated extraction of salient musical features using both signal analysis and computer vision algorithms. An audio spectrogram decomposes a recording into frequency and time, and is therefore a two dimensional image. The computer can visually analyze this image to identify regions of interest and use pattern recognition, as well as similarity features to compare them. The objective of this current research is towards the formulation of automated, computer-assisted tools for the analysis of contemporary and computer music combining the methodologies of both musicology and computer science.

This paper will give a brief overview of traditional western classical notational practices (using staff notation) in the twentieth/twenty-first century along with the aforementioned computer-assisted analysis systems (Donin, Clarke, Burleigh). Each approach has a different stance in their analysis type and how consumers should interact with this medium and the scholarship surrounding it. The author’s elaboration upon these methods and his unique approach is presented as well as a progress report on the achievements made to date. This includes the discussion of the difficulty of assigning high-level musical descriptors to abstract data sets and the examination/formulation of the nature/ontological status of this new music.

adrian 2015-06-03