Towards score following In sheet music images
M. Dorfer, A. Arzt, G. Widmer. Towards score following In sheet music images. pages 789-795, 8, 2016. | |
Autoren | |
Buch | roceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR 2016) |
Typ | In Konferenzband |
Monat | 8 |
Jahr | 2016 |
Seiten | 789-795 |
Abstract | This paper addresses the matching of short music audio snippets to the corresponding pixel location in images of sheet music. A system is presented that simultaneously learns to read notes, listens to music and matches the currently played music to its corresponding notes in the sheet. It consists of an end-to-end multi-modal convolutional neural network that takes as input images of sheet music and spectrograms of the respective audio snippets. It learns to predict, for a given unseen audio snippet (covering approximately one bar of music), the corresponding position in the respective score line. Our results suggest that with the use of (deep) neural networks – which have proven to be powerful image processing models – working with sheet music becomes feasible and a promising future research direction. |