Arctic Fracture: Drums Downtown V: Process Blog

This page documents the major developments and processes involved in the construction of the 3D animation for Drums Downtown V.
  
 

4.8.08

Video documentation is now available! Click on the video to the right to view it or you can download the video using the side menu link. (35MB flash video)

 

2.29.08 and 3.1.08

Both performances were a huge success. I hope to have complete footage from the performance in a couple weeks at the most.
 

2.28.08

This video footage is from the dress rehearsal and it was the first time this system was tested with six microphone inputs, one for each musician.

 

Each musician is responsible for a different visual element in the piece. The entire ensemble collectively controls the camera movement and direction in which the visuals are generated.

 

2.20.08

Final preparations and adjustments are being made for the performance coming next week. A short video clip demonstrates the newest cress-splash interation. The audio is an excerpt from Origins/Glades by Henrik Strindberg which is the piece that will be performed.
 

2.12.08

Additional rehearsals have yielded good results for the visualization. Controls now allow me to switch different audio inputs to different visual instruments on the fly.

 

For example, ice splashes can now be switched between the six different audio inputs based on who is playing the cymbals.

 

Adjustments have also been made to all of the instruments and camera control systems. A black screen fader has been added to allow the animation to fade in and out.

 

The newest incarnation of the cress splash is shown to the right.

 

2.1.08

The first demonstration in a live rehearsal setting was a huge success. A lot of audio level tweaking was required, but the end result was an incredible real-time visualization of their rehearsal. I have also completed the sixth instrument.

 

New control systems have also been introduced and I am working on modularizing the entire system so it is easier to switch between different mic inputs, assign different animations to different inputs, and make the overall system more flexible.

 

Click on the Flash video to right to view the latest instrument!

 

1.29.08

I have added in two more 3D elements to the animation. For a lack of better terms, they are referred to as a pike, and puffer. The brings the total instruments to 4. Icicles, crescent splash, pike, and puffer.

 

The next phase is to create the sixth instrument, and begin to design the control systems around the six different microphone inputs that will be used for the final performance.

 

1.25.08

First video footage documentation added! This is a rough concept of the motions that will be involved in the final animation. Unfortunately, there is no audio available however the animations are responding to audio. Click on the Flash video to right to view it.

 

1.21.08

The current phase of this animation involves the creation of a procedural ice texture. This texture will represent one of the instruments. The effect is similar to the ice crystals that form on a car windshield. This crystal growth will be accelerated and animated.

 

The image to the right is an example of the frost shader network in Mental Mill AE before it is exported as HLSL code.

 

1.18.08

After seeing the first rehearsal of the percussion performance, I have realized that one musician may be responsible for 3 different instruments.

 

This means that the visual instruments I created (like ice splashes for lower sounding instruments and snowflakes for cortales cymbal crashes) will not be adequate if each one is associated to just one musician. One musician may be responsible for both the cortales cymbals (high pitched sound) and a Nigerian udu earthenware drum. (lower pitched sound)

 

This means that the visuals will need to be more modular and able to shift quickly from one mic input to another. The image to the right shows a small portion of the schematic that has programmed so far.

 

1.17.08

In addition to the creation of automated systems like the camera aiming and the animation generation, I am also working on building controls that allow the artist to respond to important moments in the music that the computer controlled systems may overlook. Critical changes, larger patterns, or themes can be more easily accomodated with artist controls over the system.

 
 

1.12.08

The animation system is built around a central node. This primary node is responsible for the camera's aim and the base point for construction. Visual elements will appear relative to the position of this node.

 

The image to the right illustrates the central node (red cube) and the camera focusing on this target.

 

1.11.08

The camera system is automated according to the overall audio mix of the ensemble. As the ensemble collectively gets louder or softer, the camera will rotate around the subject matter to varying degrees and it will also zoom in or out according to the dynamics of the music.
 

1.8.08

The initial design is to associate each musician with a different visual element. Based on the water splash motion, I created a frozen splash and decided to continue with an artic visual theme. This includes snowflakes, ice splashes, and sprouting icicles. Each of these elements would expand and contract in size according to how loud or soft their instrument is played.

 

1.5.08

My concept came from watching slow motion video of water drops falling and icicles forming. Watching these small waves of water splash upwards into these beautiful arcs was the major inspirational moment. I decided to try and capture this motion and use it to represent the drum hits.
 

12.20.07

The first system was built in Max/MSP and it consists of beat detectors, audio spectrum analysis, pitch detection, and other functions that calculate the differences in sound pressure.

There are seven different units responsible for the audio analysis. Each unit can calculate all the functions mentioned and they are each tied to one of the six musicians. The seventh the unit is responsible for the overall mix or all six musicians combined.

This hierarchical structure allows me to assign various levels of control to different animations. Each musician can have their own 3D representation and the overall mix can be used to control larger animations or functions like camera movement.

Beats are not being detected with bonk. Instead, I am calculating the difference in the amplitude over 10 different frequency ranges. It is basically the same function as bonk, but it allows a little more flexibility in detecting beats that are not as pronounced.

This data is then sent over to Virtools 4.0.
 

12.14.07

This year's performance will be a real-time responsive installation. Instead of creating a pre-rendered animation that the percussionists must conform to, this animation will be generated in real-time and respond to the percussionists.