Role on this Project: Researcher
Project Website: http://phenicx.upf.edu
Modern digital multimedia and internet technology have radically changed the ways people find entertainment and discover new interests online, seemingly without any physical or social barriers. Such new access paradigms are in sharp contrast with the traditional means of entertainment. An illustrative example of this is live music concert performances that are largely being attended by dedicated audiences only. The PHENICX project aims at bridging the gap between the online and offline entertainment worlds. It will make use of the state-of-the-art digital multimedia and internet technology to make the traditional concert experiences rich and universally accessible: concerts will become multimodal, multi-perspective and multilayer digital artefacts that can be easily explored, customized, personalized, (re)enjoyed and shared among the users. The main goal is twofold: (a) to make live concerts appealing to potential new audience and (b) to maximize the quality of concert experience for everyone. Scientific objectives of PHENICX are (i) to generate a reliable and effective set of techniques for multimodal enrichment of live music concert recordings suitable for implementation and deployment in real-world situations, and (ii) finding ways to offer the resulting multi-faceted digital artefacts as engaging digital experiences to a wide range of users. The project will establish a methodological framework to map these scientific objectives onto a solid implementation platform that will be developed incrementally and tested in real-life use settings. PHENICX will mainly focus on classical music - a European heritage asset that suffers heavily from an image of inaccessibility to outsider audiences. However, findings from PHENICX will be relevant to live concert situations in any genre. With an innovative technology partner, as well as two authoritative professional music stakeholders in the consortium, the project has strong immediate impact and dissemination potential.
On this project, I was responsible for research in live tracking of orchestral music. A very robust prototypical system was publicly demonstrated at multiple occasions, the most memorable one being a demonstration at the Concertgebouw in Amsterdam, where it followed the Concertgebouw Orchestra playing Richard Strauss’ Alpensinfonie.