Other Publications

Designing and authoring gestures for socially expressive robots has been an increasingly important problem in recent years. In this demo we present a new pipeline that enables animators to create gestures for robots in a 3D animation authoring environment, without knowledge in computer programming. The pipeline consists of an exporter for a 3D animation software and an interpreter running on a System-on-Module translating the exported animation into motor control commands.

We propose an architecture that integrates Theory of Mind into a robot’s decision-making to infer a human’s intention and adapt to it. The architecture implements human-robot collaborative decision-making for a robot incorporating human variability in their emotional and intentional states. This research first implements a mechanism for stochastically estimating a human’s belief over the state of the actions that the human could possibly be executing. Then, we integrate this information into a novel stochastic human-robot shared planner that models the human’s preferred plan. Our contribution lies in the ability of our model to handle the conditions: 1) when the human’s intention is estimated incorrectly and the true intention may be unknown to the robot, and 2) when the human’s intention is estimated correctly but the human doesn’t want the robot’s assistance in the given context. A robot integrating this model into its decision-making process would better understand a human’s need for assistance and therefore adapt to behave less intrusively and more reasonably in assisting its human companion.

Wizard-of-Oz (WoZ) is a common technique enabling HRI researchers to explore aspects of interaction not yet backed by autonomous systems. A standardized, open, and flexible WoZ framework could therefore serve the community and accelerate research both for the design of robotic systems and for their evaluation.

This paper presents the definition of OpenWoZ, a Wizard-of-Oz framework for HRI, designed to be updated during operation by the researcher controlling the robot. OpenWoZ is implemented as a thin HTTP server running on the robot, and a cloud-backed multi-platform client schema. The WoZ server accepts representational state transfer (REST) requests from a number and variety of clients simultaneously. This “separation of concerns” in OpenWoZ allows addition of commands, new sequencing of behaviors, and adjustment of parameters, all during run-time.

Social presence has two opposing effects on human corruption: the collaborative and contagious nature of another person’s presence can cause people to behave in a more corrupt manner. In contrast, the monitoring nature of another person’s presence can decrease corruption. We hypothesize that a robot’s presence can provide the best of both worlds: Decreasing corruption by providing a monitoring presence, without increasing it by collusion. We describe an experimental study currently underway that examines this hypothesis, and report on initial findings from pilot runs of our experimental protocol.

Timing plays a role in a range of human-robot interaction scenarios, as humans are highly sensitive to timing and interaction fluency. It is central to spoken dialogue, with turn-taking, interruptions, and hesitation influencing both task efficiency and user affect. Timing is also an important factor in the interpretation and generation of gestures, gaze, facial expressions, and other nonverbal behavior. Beyond communication, temporal synchronization is functionally necessary for sharing resources and physical space, as well as coordinating multi-agent actions. Timing is thus crucial to the success of a broad spectrum of HRI applications, including but not limited to situated dialogue; collaborative manipulation; performance, musical, and entertainment robots; and expressive robot companions. Recent years have seen a growing interest in the HRI community in the various research topics related to human-robot timing. The purpose of this workshop is to explore and discuss theories, computational models, systems, empirical studies, and interdisciplinary insights related to the notion of timing, fluency, and rhythm in human-robot interaction.

Please refer to the new journal version of this paper.

Collaborative fluency is the coordinated meshing of joint activities between members of a well-synchronized team. We aim to build robotic team members that can work side-by-side humans by displaying the kind of fluency that humans are accustomed to from each other. As part of this effort, we have developed a number of metrics to evaluate the level of fluency in human-robot shared-location teamwork. In this paper we discuss issues in measuring fluency, present both subjective and objective metrics that have been used to measure fluency between a human and robot, and report on findings along the proposed metrics.

Music listening is a central activity in human culture, and throughout history the introduction of new audio reproduction technologies have influenced the way music is consumed and perceived.

In this work, we discuss a robotic speaker, designed to behave as both a reproduction device and as a music listening companion. The robot is intended to enhance a human’s listening experience by providing social presence and embodied musical performance. In a sample application, it generates segment-specific, beat-synchronized gestures based on the song’s genre, and maintains eye-contact with the user.

We describe an experimental human-subject study (n=67), evaluating the effect of the robot’s behavior on people’s enjoyment of the songs played, as well as on their sense of the robot’s social presence and their impression of the robot as an autonomous agent.

This paper suggests to turn to the performative arts for insights that may help the fluent coordination and joint-action timing of human-robot interaction (HRI). We argue that theater acting and musical performance robotics could serve as useful testbeds for the development and evaluation of action coordination in robotics. We also offer two insights from theater acting literature for HRI: the maintenance of continuous sub-surface processes that manifest in motor action, and an emphasis on fast, inaccurate responsiveness using partial information and priming in action selection.

Shimon is an autonomous marimba-playing robot designed to create interactions with human players that lead to novel musical outcomes. The robot combines music perception, interaction, and improvisation with the capacity to produce melodic and harmonic acoustic responses through choreographic gestures. We developed an anticipatory action framework, and a gesture-based behavior system, allowing the robot to play improvised Jazz with humans in synchrony, fluently, and without delay. In addition, we built an expressive non-humanoid head for musical social communication. This paper describes our system, used in a performance and demonstration at the CHI 2010 Media Showcase.

Theater actors have been staging artificial intelligence for centuries. If one shares the view that intelligence manifests in behavior, one must wonder what lessons the AI community can draw from a practice that is historically concerned with the infusion of artificial behavior into such vessels as body and text. Like researchers in AI, actors construct minds by systematic investigation of intentions, actions, and motor processes with the proclaimed goal of artificially recreating human-like behavior. Therefore, acting methodology may hold valuable directives for designers of artificially intelligent systems. Indeed, a review of acting method literature reveals a number of insights that may be of interest to the AI community.

A mounting body of evidence in psychology and neuroscience points towards an embodied model of cognition, in which the mechanisms governing perception and action are strongly interconnected, and also play a central role in higher cognitive functions, traditionally modeled as amodal symbol systems.

We argue that robots designed to interact fluidly with humans must adopt a similar approach, and shed traditional distinctions between cognition, perception, and action. In particular, embodiment is crucial to fluid joint action, in which the robot’s performance must tightly integrate with that of a human counterpart, taking advantage of rapid sub-cognitive processes.

We thus propose a model for embodied robotic cognition that is built upon three propositions: (a) modal, perceptual models of knowledge; (b) integration of perception and action; (c) top-down bias in perceptual processing. We then discuss implications and derivatives of our approach.

We aim to build robots that go beyond command-and-response and can engage in fluent collaborative behavior with their human counterparts. This paper discusses one aspect of collaboration fluency: expectation management -predicting what a human collaborator will do next and how to act on that prediction. We propose a formal time-based collaborative framework that can be used to evaluate this and other aspects of collocated human-robot teamwork, and show how expectation management can enable a higher level of fluency and improved efficiency in this framework. We also present an implementation of the proposed theoretical framework in a simulated human-robot collaborative task.