Gesture-Based Character Controller

[January 8, 2007 - February 15, 2007]

In the Winter Quarter of 2006 – 2007 I was experimenting with Virtools and the 5-D Dataglove in Alan Price’s Buliding Virtual Environment’s class I used one of Alan’s custom Virtools Plug-ins called the Glove BB to interface with Virtools. I kept a process blog for this project, so feel free to check it out if you’d like to learn more.


The Dataglove sends information through the serial port regarding the pitch and roll of the wrist as well as the bend of each finger. In addition a series of 15 hand gestures are recognized and corresponding ‘gesture codes’ are sent. For example: a closed fist would send a zero and pointing with the index finger only would send a twelve.

eva walking

I developed a system that allows you to control a virtual character named Eva (part of Virtools existing resources) using somewhat modified American Sign Language for run, walk and sit. Eva is also programed to jump and open doors when certain gestures are performed with the dataglove, although these hand gestures are more kinetographic and do not relate in any way to American Sign. When I demonstrated this project at ACCAD’s annual open house in 2007, I demonstrated the sign signals and people could quickly catch on. It seemed like a fairly intuitive way of stearing the character, although my arm got pretty tired after a few hours of demonstration.

joint hierarchy

I took the Eva project a bit further by creating a system of navigation that allows you to move through a character’s joint hierarchy using pointing and thumbing gestures in combinations with wrist bend and rotation. The end product was a little demo of how one might be able to do pose to pose animation with a data-glove. The final project was featured breifly on FOX 28's Good Day Columbus. I plan to post this video and a full demo of these projects sometime in the near future, so keep a lookout!