Final Report

Researchers in the Human-Robot Interaction (HRI) community have addressed a spectrum of challenges at the intersection of robotics, social and cognitive psychology, human factors, and AI.  At the same time, progress is being made in the Spoken Dialog community including work on the development of fundamental theories, formalisms, and models. To date, spoken dialog efforts have focused almost exclusively on applications within restricted communication contexts, such as telephone- and PC-based information access. Several research efforts at the intersection of Spoken Dialog and HRI have sought to broaden dialog to richer, more natural settings. These efforts have identified numerous challenges with the use of dialog as part of coordination among multiple actors, taking into consideration details of the tasks at hand and the surrounding environment.

The AAAI Fall Symposium on Dialog with Robots was organized to catalyze communication and innovation at the crossroads of Spoken Dialog and HRI.  The meeting brought together over 70 researchers from the HRI, spoken dialog systems, intelligent virtual agents, and other related research communities in an open discussion about the challenges at the intersection of these fields. The proceedings contain over 40 contributions.  Papers from the proceedings, participants, and related information can be found on this website.

Ideas spanning a spectrum of interrelated research topics were presented and discussed during oral presentations and a poster session.  Recurrent themes centered around challenges and directions with the use of dialog by physically embodied agents, taking into consideration aspects of the task, surrounding environment, and broader context.  Several presentations highlighted problems with modeling communicative competencies that are fundamental in creating, maintaining and organizing interactions in physical space, such as engagement, turn-taking, joint attention, and verbal and non-verbal communicative skills. Other presenters explored the challenges of leveraging physical context in various language understanding problems such as reference resolution, or the challenges of coupling action and communication in the interaction planning and dialog management process. A number of papers reported on developmental approaches for acquiring knowledge through interaction, and focused on challenges such as learning new words, concepts, meanings and intents, and grounding this learning in the interaction with the physical world. The topics covered also included interaction design challenges, descriptions of existing or planned systems, research platforms and toolkits, theoretical models, and experimental results.

Invited keynote presentations provided different perspectives on work in dialog with robots and complemented views and themes arising at the symposium. Cynthia Breazeal outlined progress in recent years in the HRI community, including developments in social robotics and uses of models of mind in robotic systems. Candace Sidner provided a review of the state-of-the-art on modeling discourse and dialog in the Spoken Dialog community and highlighted challenges in pushing the boundaries of the current models at different interaction time-scales, from seconds to hours to months. Herbert H. Clark added a valuable psycholinguistics perspective to the meeting by sharing his reflections about how people coordinate joint activities. He presented results from studies with human subjects on the timing of utterances during collaborations, highlighting sets of competencies required for fluid spoken interaction and collaboration.

In addition to the technical and keynote presentations, the symposium included three moderated, open discussions that provided a forum for exchanging ideas on some of the key topics in the symposium. The first discussion aimed to address some of the challenges at the crossroads of dialog and HRI. The physicality of such interactions was highlighted as a critical factor and the prospect of identifying a core, yet simple set of principles and first-order concepts to be reasoned about, or a “naïve physics” of situated dialog and discourse, was raised and discussed. A second open discussion centered around the interplay between action and communication, and highlighted ideas such as viewing communication as joint action and the importance of creating models for incremental processing that can support recognition and generation of actions and phenomena occurring on different time scales. The final discussion addressed several other fundamental issues such as how we might move forward in this nascent field.  Discussion touched on the need for unified platforms and challenges for supporting comparative evaluations of different techniques, the pros and cons of simulation-based approaches, and even the value of revisiting fundamental questions: Why should we endow robots with the ability to engage in dialogue with people?  What assumptions are we making–and which can we make?

Altogether, the density of ideas captured in the technical contributions and participants, the animated discussions, and the diversity of questions raised, and the different technical approaches taken reflect a nascent, vibrant community unified by the common themes of dialog with robots or interaction within a physical context. We thank the authors and participants for their valuable contributions and perspectives on these problems and hope that the momentum generated by this Symposium will carry forward and help to catalyze future research efforts and meetings.

Dan BohusEric HorvitzTakayuki KandaBilge Mutlu, and Antoine Raux served as co-chairs of this Symposium. The papers of the Symposium are published as AAAI Press Technical Report FS-10-01.