Showcase

In my research, I place a special focus on live demonstrations of my results to researchers, musicians and the general public. On this page, I have collected video impression of some of these events.

The Piano Tracker: An Automatic Page-turner

This video shows a demonstration of the piano tracker at the Vienna Talk on Musical Acoustics 2010. Werner Goebl is playing the piano. The system is “listening” to the audio and “reading” along in the sheet music. At the appropriate moments, it triggers a device to automatically turn the pages for the musician.

Near-instant Music Identification for Piano Music

This video shows an algorithm that is able to almost instantly identify the piece a performer is playing on the piano. To do that, it has to match the live audio to a large database of symbolic scores. This is done via a generalised fingerprinting algorithm.

The Piano Music Companion: Live Demonstration at the ICT 2013

By combining the identification and the tracking technology, we built a flexible and robust music tracking system which is able to follow and understand (at least to some extent) a live piano performance. Within a few seconds this system can identify the piece that is being played, and the position within the piece. It then tracks the progress of the performer over time via a robust score following algorithm. Furthermore, the system continuously re-evaluates its current position hypotheses within a database of scores (roughly 1,000,000 notes!) and is capable of detecting arbitrary ‘jumps’ by the performer (e.g., leaving out repetitions, re-starts at any position, etc.) – not only within a piece, but within the complete database of classical piano scores.

The Piano Music Companion: Decomposing Mozart

Cynthia Liem is stress-testing the system by butchering (on purpose) one of Mozart’s most well known pieces.

The Orchestra Follower

This is an extension of the piano tracker to orchestral music. The video gives an impression of a live experiment at the Concertgebouw in Amsterdam. It listens to the live music and follows the progress in the sheet music, i.e., continuously tracks the most likely position of the live music in the printed score. his information, in turn, is used to enrich the concert experience for members of the audience by stream- ing synchronised visual content (the sheet music, explanatory text and videos) onto tablet computers in the concert hall.