Summary

Interactive training application incorporating an artificial intelligence-driven, customizable avatar with a conversation engine. The application is to be used for medical students preparing for examinations involving new patients.

Project Team

Faculty
Dr. Doug Danforth, Obstetrics and Gynecology, OSU Medical Center
Alan Price, Associate Professor, Design/ACCAD

Staff
Vita Berezina-Blackburn (Motion Capture) ACCAD

Graduate Students
Kellen Maicher, Graduate Research Associate, Design/ACCAD
Jeremy Baker, Graduate Research Associate, Design/ACCAD (Modeling Contributor)
Thomas Heban, Graduate Research Associate, Design/ACCAD (Motion-Capture Contributor)

Project Description

An interactive training application incorporating an artificial intelligence-driven, customizable avatar with a conversation engine. The application is to be used for medical students preparing for examinations involving interviews with new patients.

The research incorporates affective virtual character design for agency between doctor and patient. Emotion-driven responses from the avatar are integrated with spoken answers to the doctor's questions. The conversation engine server associates emotions with questions related to the medical case created for the patient interview.

Procedural animation programming drives facial expressions, generating real-time lip-sync matched to voice synthesis of text retrieved from the conversation engine database, and postures of the patient avatar. Voice recognition and gesture-based motion tracking allow the student doctor to interact naturally with the avatar patient, including eye contact. An immersive “sim-room” was created with a large-scale display system that provides a life-size scale simulation of the exam room environment. A desktop version allows for students to interact with the patient avatar using keyboard input for a text-based conversation.

Project currently in progress. Preliminary testing began in January 2013 with testing continuing throughout the Spring semester.