Real Time Music Visualization: A Study in the Visual Extension of Music |
|
My name is Matthew Bain and I have recently completed my graduate degree in Design at The Ohio State University. I received by Bachelor's degree in computer graphics and music. I continue to play the trumpet, piano, and compose music in my spare time. My primary research focus is in creating real-time systems that function as visual extensions of music. My research includes abstract animations of music pre-rendered in Maya, 3D visualizations of music theory structures, and real-time systems that visually react to live music performances. My original concept to marry 3D computer graphics with music started as my BFA senior show, Sculpting Music. This series of three animations incorporated my first attempts of using audio data to drive the generative components of a 3D animation. |
 |
The second phase involved the visualization of a complex structure inherent to Bach's Fugue 14 in F-sharp minor. This project illustrated the principle theme and relationships between note intervals for the first six measures of the fugue. The third phase of my research was based on the concepts from my senior BFA show. Instead of using audio data to drive an animation, I studied the use of MIDI data to control animations. Since the exact notes and timing could be extracted from MIDI, this provided more musical elements that could be mapped to visual animations. The Drums Downtown IV piece was a culmination of this research. |
|
The fourth phase incorporates aspects from all the previous projects but introduces a real-time component to the visualizations. Instead of creating a musical analysis and then rendering a movie to accompany the piece, the animation is generated in real-time based on microphone inputs and the live musical performance.
The computer’s high-speed generative capabilities are combined with the artist’s conceptual sensibilites through a hierarchical system of controls. The artist can respond to larger thematic and stylistic changes in the music while the computer manages detailed audio analysis and generates the visual forms in the 3D animation. The resulting scenario demonstrates the artist as a visual contributing member to the music ensemble, and the computer’s role serves as a visual instrument for the artist. I intend to continue exploring this research and generate 3D animations with live musical performances. In some sense, I am trying to automate Disney's Fantasia for a live musical performance. |
 |
If you're interested in using my work, would like to see a visualization of a particular piece of music, or would like to collaborate on a live performance-based piece, please don't hesitate to contact me.
|
|
