Towards score following In sheet music images
|M. Dorfer, A. Arzt, G. Widmer. Towards score following In sheet music images. pages 789-795, 8, 2016.|
|Buch||roceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR 2016)|
This paper addresses the matching of short music audio snippets to the corresponding pixel location in images of sheet music. A system is presented that simultaneously learns to read notes, listens to music and matches the currently played music to its corresponding notes in the sheet. It consists of an end-to-end multi-modal convolutional neural network that takes as input images of sheet music and spectrograms of the respective audio snippets. It learns to predict, for a given unseen audio snippet (covering approximately one bar of music), the corresponding position in the respective score line. Our results suggest that with the use of (deep) neural networks – which have proven to be powerful image processing models – working with sheet music becomes feasible and a promising future research direction.