Thesis work and documentation

Matthew Bain

Real-time visualizations of music need to adapt on-the-fly as the music changes. Musicians can even break out into improvisational phrases which can further complicate things. MIDI provides a good framework for analysis but during a live performance, it is difficult for a computer to follow a musician while still remaining within the structure of MIDI.

To combat these difficulties, I have taken the approach of analyzing audio waveforms from microphone inputs. This allows the computer to respond to the musician's performance without being constrained to any framework.

Two other areas that this project intersects with are the Mapping of Sound page and the MEL scripting and analysis page.

These visualizations incorporate the use of Max/MSP to provide real-time audio analysis from multiple microphone sources. So in theory, an orchestra could have multiple microphones for different sections and various analyses could be performed on each of those sections.

The prototypes for this project incorporated the use of particle systems and small colored spheres. The first example used various audio parameters like amplitude, pitch, and beat detection to control parameters in a particle system.

The goal is to eventually construct a visualization system that can capture patterns in music. Additional information on visually mapping sound is available here.

Below are some screenshots of the current prototype. The image at the very bottom is actually a pattern that was accurately illustrated based on the repetitious nature of the song that was playing at the time. As you can see, there are 3 distinct phrases that reflect the musical pattern.