Papers in Refereed Conferences

With domestic technology on the rise, the quantity and complexity of smart-home devices are becoming an important interaction design challenge. We present a novel design for a home control interface in the form of a social robot, commanded via tangible icons and giving feedback through expressive gestures. We experimentally compare the robot to three common smart-home interfaces: a voice-control loudspeaker; a wall-mounted touch-screen; and a mobile application. Our findings suggest that interfaces that rate higher on flow rate lower on usability, and vice versa. Participants’ sense of control is highest using familiar interfaces, and lowest using voice control. Situation awareness is highest using the robot, and also lowest using voice control. These findings raise questions about voice control as a smart-home interface, and suggest that embodied social robots could provide for an engaging interface with high situation awareness, but also that their usability remains a considerable design challenge.

Human-robot collaboration will increasingly take place in human social settings, including contexts where ethical and honest behavior is paramount. How might these robots affect human honesty? In this paper, we present first evidence of how a robot’s presence affects people’s ethical behavior in a controlled field study. We observed people passing by a food plate marked as “reserved”, comparing three conditions: no observer, a human observer, and a robot observer. We found that a human observer elicits less attention than a robot, but evokes more of a socially normative presence causing people to act honestly. Conversely, we found that a robot observer elicits more attention, engagement, and a monitoring presence. But even though people were suspicious that they were being monitored, they still behaved dishonestly in the robot observer condition.

We describe the design process of  “Vyo”, a personal assistant serving as a centralized interface for smart home devices. Building on the concepts of ubiquitous and engaging computing in the domestic environment, we identified five design goals for the home robot: engaging, unobtrusive, device-like, respectful, and reassuring. These goals led our design process, which included simultaneous iterative development of the robot’s morphology, nonverbal behavior and interaction schemas. We continued with user-centered design research using puppet prototypes of the robot to assess and refine our design choices. The resulting robot, Vyo, straddles the boundary between a monitoring device and a socially expressive agent, and presents a number of novel design outcomes: The combination of TUI “phicons” with social robotics; gesture-related screen exposure; and a non-anthropomorphic monocular expressive face. We discuss how our design goals are expressed in the elements of the robot’s final design.

Responsiveness to one’s bids for proximity in times of need is a linchpin of human interaction. As such, the ability to be perceived as responsive has design implications for socially assistive robotics. We report on a large-scale experimental laboratory study (n=102) examining robot responsiveness and its effects on human attitudes and behaviors. In one-on-one sessions, participants disclosed a personal event to a non-humanoid robot. The robot responded either responsively or unresponsively across two modalities: Simple gestures and written text. We replicated previous findings that the robot’s responsiveness increased perceptions of its appealing traits. In addition, we found that robot responsiveness increased nonverbal approach behaviors (physical proximity, leaning toward the robot, eye contact, smiling) and participants’ willingness to be accompanied by the robot during stressful events. These findings suggest that humans not only utilize responsiveness cues to ascribe social intentions to personal robots, but actually change their behavior towards responsive robots and may want to use such robots as a source of consolation.

We present the design and initial evaluation of Kip3, a social robotic device for students with ADHD that provides immediate feedback for inattention or impulsivity events. We designed a research platform comprised of a tablet-based Continuous Performance Test (CPT) that is used to assess inattention and impulsivity, and a socially expressive robotic device (Kip3) as feedback. We evaluated our platform with 10 students with ADHD in a within subject user study, and report that 9 out of 10 participants felt that Kip3 helped them regain focus, but wondered if it will be effective over time and how it will identify inattention in more complex situations outside the lab.

We evaluate the effects of robotic listening companionship on people’s enjoyment of music, and on their perception of the robot. We present a robotic speaker device designed for joint listening and embodied performance of the music played on it. The robot generates smoothed real-time beat-synchronized dance moves, uses nonverbal gestures for common ground, and can make and maintain eye-contact.

In an experimental between-subject study (n=67), participants listened to songs played on the speaker device, with the robot either moving in sync with the beat, moving off-beat, or not moving at all. We found that while the robot’s beat precision was not consciously detected by Ps, an on-beat robot positively affected song liking. There was no effect on overall experience enjoyment. In addition, the robot’s response caused Ps to attribute more positive human-like traits to the robot, as well as rate the robot as more similar to themselves. Notably, personal listening habits (solitary vs. social) affected agent attributions.

This work points to a larger question, namely how a robot’s perceived response to an event might affect a human’s perception of the same event.

Combining high-performance, sensor-rich mobile devices with simple, low-cost robotic platforms could accelerate the adoption of personal robotics in real-world environments. We present a case study of this “dumb robot, smart phone” paradigm: a robotic speaker dock and music listening companion. The robot is designed to enhance a human’s listening experience by providing social presence and embodied musical performance. In its initial application, it generates segment-specific, beat-synchronized gestures based on the song’s genre, and maintains eye-contact with the user. All of the robot’s computation, sensing, and high-level motion control is performed on a smartphone, with the rest of the robot’s parts handling mechanics and actuator bridging.

Shimon is a interactive robotic marimba player, developed as part of our ongoing research in Robotic Musicianship (RM). One of the potential benefits of RM is that it provides human players with embodied information that relates spatial movement to tone generation. This can aid in anticipation and coordination of synchronous playing.

As part of a human-robot Jazz improvisation system, we present an anticipatory system enabling beat-matched real-time synchronization. Our system enables flexible, yet coordinated call-and-response, a standard type of musical interaction. It was used in a live public human-robot joint Jazz performance.

We also describe a preliminary study evaluating the effect of embodiment on this call-and-response musical synchronization task. We conducted a 3×2 within-subject study manipulating the level of embodiment (visual co-presence, physical presence but visual occlusion, and synthesized sound), and the accuracy of the robot’s response.

Our findings indicate that synchronization is aided by visual contact when uncertainty is high, but that pianists can resort to internal rhythmic coordination in more predictable settings. We find that visual coordination is more effective for synchronization for slow sequences compared to faster sequences; and that occluded physical presence may be less effective than audio-only note generation.

This paper describes a robotic puppeteering system used in a theatrical production involving one robot and two human performers on stage. We draw from acting theory and human-robot interaction to develop a hybrid-control puppeteering interface which combines reactive expressive gestures and parametric behaviors with a point-of-view eye contact module. Our design addresses two core considerations: allowing a single operator to puppeteer the robot’s full range of behaviors, and allowing for gradual replacement of human-controlled modules by autonomous subsystems.

We wrote a play specifically for a performance between two humans and one of our research robots, a robotic lamp which embodied a lead role in the play. We staged three performances with the robot as part of a local festival of new plays. Though we have yet to perform a formal statistical evaluation of the system, we interviewed the actors and director and present their feedback about working with the system.

We have developed a cognitive architecture for a robotic teammate based on the neuro-psychological principles of anticipation and perceptual simulation, with the aim of attaining increased fluency and efficiency in human-robot teams. An instantiation of this architecture was implemented on a non-anthropomorphic robotic lamp, performing in a human-robot collaborative task.

In a human-subject study in which the robot works on a joint task with untrained subjects, we find our approach to be significantly more efficient and fluent than in a comparable system without anticipatory perceptual simulation. We also show the robot and the human to be increasingly contributing at a similar rate, and we find significant differences in a number of self-report measures and verbal attitudes towards the robot. Notably, we identify increased self-deprecation in human subject responses vis-à-vis the anticipatory robot.

We have developed a cognitive architecture for robotic teammates based on the neuro-psychological principles of perceptual symbols and simulation, with the aim of attaining increased fluency in human-robot teams. An instantiation of this architecture was implemented on a robotic desk lamp, performing in a human-robot collaborative task. This paper describes initial results from a human-subject study measuring team efficiency and team fluency, in which the robot works on a joint task with untrained subjects. We find significant differences in a number of efficiency and fluency metrics, when comparing our architecture to a purely reactive robot with similar capabilities.

A crucial skill for fluent action meshing in human team activity is a learned and calculated selection of anticipatory actions. We believe that the same holds for robotic teammates, if they are to perform in a similarly fluent manner with their human counterparts. In this work, we propose an adaptive action selection mechanism for a robotic teammate, making anticipatory decisions based on the confidence of their validity and their relative risk. We predict an improvement in task efficiency and fluency compared to a purely reactive process.

We then present results from a study involving untrained human subjects working with a simulated version of a robot using our system. We show a significant improvement in best-case task efficiency when compared to a group of users working with a reactive agent, as well as a significant difference in the perceived commitment of the robot to the team and its contribution to the team’s fluency and success. By way of explanation, we propose a number of fluency metrics that differ significantly between the two study groups.

Time Bracketing is a novel technique to photographically depict a time-varying space or object in a single image without spatial distortion or fragmentation. The approach attempts to strike a balance between procedural generation and manual composition as well as between faithful depiction and digital manipulation. It also enables the clear representation of both dynamic and static scene elements. As a result, Time Bracketing puts the subject, rather than the technique, in the center of its artistic creation. This paper introduces the method, presents a custom-built authoring software for the creation of Time Bracketing images, and shows and architectural studies using the described technique and software.

While Reinforcement Learning (RL) is not traditionally designed for interactive supervisory input from a human teacher, several works in both robot and software agents have adapted it for human input by letting a human trainer control the reward signal. In this work, we experimentally examine the assumption underlying these works, namely that the human-given reward is compatible with the traditional RL reward signal. We describe an experimental platform with a simulated RL robot and present an analysis of real-time human teaching behavior found in a study in which untrained subjects taught the robot to perform a new task.

We report three main observations on how people administer feedback when teaching a robot a task through Reinforcement Learning: (a) they use the reward channel not only for feedback, but also for future-directed guidance; (b) they have a positive bias to their feedback — possibly using the signal as a motivational channel; and (c) they change their behavior as they develop a mental model of the robotic learner. In conclusion, we discuss future extensions to RL to accommodate these lessons.

In Socially Guided Machine Learning we explore the ways in which machine learning can more fully take advantage of natural human interaction. In this paper we are studying the role real-time human interaction plays in training assistive robots to perform new tasks. We describe an experimental platform, Sophie’s World, and present descriptive analysis of human teaching behavior found in a user study. We report three important observations of how people administer reward and punishment to teach a simulated robot a new task through Reinforcement Learning. People adjust their behavior as they develop a model of the learner, they use the reward channel for guidance as well as feedback, and they may also use it as a motivational channel.

Many new applications for robots require them to work alongside people as capable members of human-robot teams. These include—in the long term—robots for homes, hospitals, and offices, but already exist in more advanced settings, such as space exploration. The work reported in this paper is part of an ongoing collaboration with NASA JSC to develop Robonaut, a humanoid robot envisioned to work with human astronauts on maintenance operations for space missions. To date, work with Robonaut has mainly investigated performing a joint task with a human in which the robot is being teleoperated. However, perceptive disorientation, sensory noise, and control delays make teleoperation cognitively exhausting even for a highly skilled operator. Control delays in long range teleoperation also make shoulder-to-shoulder teamwork difficult. These issues motivate our work to make robots collaborating with people more autonomous.

Our work focuses on a scenario of a human and an autonomous humanoid robot working together shoulder-to-shoulder, sharing the workspace and the objects required to complete a task. A robotic member of such a team must be able to work towards a shared goal, and be in agreement with the human as to the sequence of actions that will be required to reach that goal, as well as dynamically adjust its plan according to the human’s actions. Human-robot collaboration of this nature is an important yet relatively unexplored kind of human-robot interaction.

This paper describes our work towards building a dynamic collaborative framework enabling such an interaction. We discuss our architecture and its implementation for controlling a humanoid robot, working on a task with a human partner. Our approach stems from Joint Intention Theory, which shows that for joint action to emerge, teammates must communicate to maintain a set of shared beliefs and to coordinate their actions towards the shared plan. In addition, they must demonstrate commitment to doing their own part, to the others doing theirs, to providing mutual support, and finally—to a mutual belief as to the state of the task.

We argue that to this end, the concept of task and action goals is central. We therefore present a goal-driven hierarchical task representation, and a resulting collaborative turn-taking system, implementing many of the above-mentioned requirements of a robotic teammate. Additionally, we show the implementation of relevant social skills supporting our collaborative framework.

Finally, we present a demonstration of our system for collaborative execution of a hierarchical object manipulation task by a robot-human team. Our humanoid robot is able to divide the task between the participants while taking into consideration the collaborator’s actions when deciding what to do next. It is capable of asking for mutual support in the cases where it is unable to perform a certain action. To facilitate this interaction, the robot actively maintains a clear and intuitive channel of communication to synchronize goals, task states, and actions, resulting in a fluid, efficient collaboration.