iPhone as Musical Instrument from UMNewsService on Youtube.
For the last year Georg Essl from the University of Michigan has been working on an interactive programming environment for mobile devices. An alpha version of urMus, currently used by the Stanford Mobile Phone Orchestra (MoPhO) and Michigan Mobile Phone Ensemble, has recently been released for download. I was lucky enough to be invited to the second urMus workshop ever, held by Essl on the 21 June 2010 at the University of Wollongong.
Basically urMus is a tool for the creation of sonic and visual instruments directly on mobile devices (currently the iPhone and iPad, with wider non-Apple compatibility coming in the future). It gives you multiple entrance points from which to build these instruments. You can use Essl’s default urMus interface to work in a high level Pd-like way linking boxes together to create and process sound and vision, or you can delve deeper, using urMus lua code to build your own processes and interfaces. Where it differs from apps like RjDj is that it lets you create your own instruments and interfaces directly on the mobile device itself.
urMus makes full use of the capabilities of mobile devices including multitouch, microphone input, speaker output and accelerometer data. Essl’s default interface has been created with live coding in mind and as such everything is very accessible and controllable in a live context. For instance it takes almost no time at all to get the X-axis of an iPhone accelerometer to control the pitch of an oscillator and send that to the iPhone’s line output. You simply touch the ‘Source’ arrow, scroll through a list of flowboxes (similar to a list of Pd objects) and drag an ‘Accel X’ onto the interface’s grid. Then you touch the ‘Filter’ arrow and drag a ‘SinOsc Freq’ onto the grid. Finally you touch the ‘Sink’ arrow and drag a ‘Dac In’ onto the grid. There is no need to link up patch cords or type in object names or parameters, everything happens automatically. In about six screen touches you have a fully interactive instrument in the palm of your hand. If you then drag a ‘Vis In’ object onto the grid you instantly have an audiovisual instrument. As well as hearing a sine wave you see the wave being drawn on the screen. Tilting the iPhone on its X axis changes the frequency of both the audio and visual representation of the wave. Replace the source flowbox with a ‘Mic’ object and you have a simple pitch shifter, controlling the frequency of the sine wave with the volume of your voice. These are just a few examples of how you can use urMus to build complex instruments during a live performance without having to go anywhere near a laptop or even access the iPhone’s virtual keyboard.
A key feature of urMus though is that you aren’t forced to work in this way. If you don’t like the way that the default interface works, you can change it to work however you want it to. Other examples of interfaces built by Essl and his students include virtual keyboards, ocarinas and samplers. If you prefer text based live coding you can even do away with an interface altogether and create audiovisual works using straight urMus lua code.
At the present time urMus is very much in alpha but is definitely worth experimenting with. I managed to crash it a few times during the workshop but when it did work it worked fast and intuitively. urMus is a clear step forward in using mobile devices not only as laptop controllers but as compact self-contained instruments in their own right, something I look forward to more of in the future.
Tags: Georg Essl, iPad, iPhone, lua, Michigan Mobile Phone Ensemble, Mobile Device, Pure Data, Stanford Mobile Phone Orchestra, University of Michigan, University of Wollongong, urMus