Improving Collaborative Interaction with Virtual Characters

Improving Collaborative Interaction with Virtual Characters

Successful collaboration relies on the coordination and align-ment of communicative cues. In this paper, we present mecha-nisms ofbidirectional gaze—the coordinated production anddetection of gaze cues—by which a virtual character can co-ordinate its gaze cues with those of its human user. We im-plement these mechanisms in a hybrid stochastic/heuristicmodel synthesized from data collected in human-human in-teractions. In three lab studies wherein a virtual characterinstructs participants in a sandwich-making task, we demon-strate how bidirectional gaze can lead to positive outcomes inerror rate, completion time, and the agent’s ability to producequick, effective nonverbal references. The first study involvedan on-screen agent and the participant wearing eye-trackingglasses. The second study demonstrates that these positiveoutcomes can be achieved using head-pose estimation in placeof full eye tracking. The third study demonstrates that theseeffects also transfer into virtual-reality interactions.



Sean Andrist, Michael Gleicher, and Bilge Mutlu


Posted on

February 14, 2017