Gesture Mapping for Interaction Design: An Investigative Process for Developing Interactive Gesture Libraries

ABSTRACT

Gestures play important roles as facilitators of language development, temporal-spatial learning, and non-verbal communication. Gesture-based interaction design seeks to capitalize on this natural method of human communication by using gestures as a means of interfacing with digital content. While technological factors address important issues related to sensing gestural input, design factors are the most critical factors relate to developing useful and approachable gesture-based interactivity. The goal of this research is to articulate more clearly some intrinsic characteristics of gesture that are significant to gestural interface designers, while providing methodologies that designers can use to gather and implement this information in a fashion that suits their unique design processes.

Gesture researchers have published a great deal of research that has significant implications related to gestural interface design, but most research in the field of gesture studies relates to gestures that are produced in combination with speech or in place of speech. Directly applying this research to visual interface design is difficult because many of the examples of gestures provided by these researchers analyze gesture in terms their linguistic characteristics. Because interface designers are seeking gestures that can be incorporated into interactive scenarios, there is a need for example of gestures produced in response to visual-spatial cues. The aim for this study and beyond is to create a library of gestures that can serve as a reference to designers who are seeking visual-spatial representations of a broad range of gestural expression. This study presents methods of visual and spatial contextualization that can be applied or expanded upon by gestural interface designers who are seeking to build unique gestural vocabularies on a project-by-project basis. This document outlines a pragmatic approach to gestural interface design that aims to inspire designers toward further investigation.

This thesis documents the development processes for several interactive prototypes. Each of these prototypes helped to define specific research questions that may be important issues as gesture-based interaction design moves forward as a field of research. Discussion of interactive prototypes is followed by documentation of a user centered research study. This study presents new strategies for provoking, documenting, analyzing and contextualizing gestures within specialized visual-spatial scenarios.

The results of this study include documentation of an approach that can be used to generate libraries of interactive gestures. Several categorical patterns of gestural expression emerged from this research study, which reaffirms the potential for standardization of gestural interaction. Because gestures have recognizable visual and formal relationships to the things that they represent, their interpretation is closely tied to the context in which they are used. Through a process of contextualization, interaction designers can create visual-spatial frameworks for understanding the intended meaning of the gestures that a user produces. This thesis discusses best practices for applying gestures within interactive scenarios by defining many characteristics of gesture that represent a broad range of gestural expression.

Thesis Proposal [thesis_proposal.pdf]

Thesis Defense [thesisDefense.pdf]

Thesis [gestureMapping.doc]
last updated June 4, 2009

Please feel free to email me with comments and suggestions