Background

For the 2018 Cork Jazz festival we turned a select number of cabs into instruments that would affect the jazz music playing, in real-time, based on their movement.

Each cab was fitted with a mobile phone that was running an app I created. I lead the developement of the app. It was built using Angular and Ionic. The app had two functions. To send the location of the cab to our server for processing and visualisation, and to allow passengers to listen in on the music they were helping to create.

When the paths of two or more taxis crossed, our server generated and emitted a trigger. Our specially developed desktop app converted this trigger into a midi-signal which allowed the composition to evolve, using audio sequencing software. One piece of evolving music that progresses over 16 hours. Each time the signal changed, there were 1000’s of possible variations. Our AI conductor, taught using machine-learning, made sure everything was in time and in key.

A dynamic WebGL visualisation was created to accompany the jazz. The audio-visual experience was streamed live on youtube and online radio station, Mixlr.

Challenges

There were a number of challenges to overcome for this project.

Firstly, we needed a number of mobile phones to use. It was a balancing act between costs and performance over what phone we chose. We needed a phone that would allow us to rapidly develop and deploy apps to. This excluded iPhones and left us with Android. We chose the Nokia 1 as it fit the criteria.

The application I developed was initially too accurate. It was sending location updates at a rapid rate which when multiplied over 5 or 10 cabs, quickly became an issue for our server to deal with. I tuned the frequency at which it was sending updates to give alleviate the strain on the server but without sacrificing on the accuracy.

View one of the live streams here: