With the free app “INSIDE MPHIL – St. Nikolai” you can experience the Munich Philharmonic Orchestra virtually on the meadow behind the church of St. Nikolai near the Munich Gasteig as if the musicians were set up there. With the GPS-connected smartphone and headphones you enter the orchestra acoustically and hear the instruments as if you were standing next to them, just as the musicians hear themselves and their colleagues: the violins, the oboe, the timpani or whatever you want to hear. This is made possible by the sophisticated smartphone app by sound designer Mathis Nitschke and violist Gunter Pretzel.
Smartphone (Android: from Samsung Galaxy S7 or similar / iOS: from iPhone SE) and headphones. Download the app “INSIDE MPHIL – St. Nikolai” from the Google Play Store or Apple App Store in advance.
To download the app please click these banners:
Where does “Inside MPhil – St. Nikolai” play?
The church St. Nikolai is located in Haidhausen, Munich, in the immediate neighbourhood of the Gasteig. The meadow where the virtual orchestra plays can be found, coming from the Gasteig, on the back side of the church, between the church and the children’s playground. You can also use this Google Link: https://goo.gl/maps/jBb8xeQX3WYYCnB59
At the heart of the app are the Spatial Sound algorithms from Dear Reality. The so called synthetic binauralization locates sound sources in the headphones coming from all directions of the three-dimensional space around the head. In other words, you can not only control whether you hear the sound from the right or left, but also from the front or the back or up or down.
Orchestras are recorded next to the main microphones with many spot microphones close to the musicians. We take all these spot microphones for the app and distribute them geographically on the meadow. You then move as a walking listener through these spot microphones, whereby all instruments can be heard by synthetic binauralization from the right direction at the right place. Your smartphone plays a total of 22 tracks, all through these HRTF filters. It’s really amazing what today’s smartphones can do.
See how much conductor Pablo Heras-Casado enjoyed the experience:
This video serves archival documentary purposes. Audio recorded directly (live) from the smartphone. Use Headphones for hearing the spatial audio. English subtitles will be added later.
Unfortunately the GPS system worked really weak on this day, reactions to positional changes were really slow. Also the GPS coordinates were offset as a whole. Usually the woodwinds are in the bushes.
Artistic research with the Munich Symphonic Orchestra
Basis for this app is a research project with the Munich Symphonic Orchestra, read about it here >>>
Are there any other related works?
Vergehen is an opera that you walk.