Data Analysis Results

In the first iteration of the SERA Field Study, participants could only use yes/no buttons for input. In the second and third iterations, the Nabaztag’s inbuilt RFID (Radio Frequency Identification) reader was used. Participants received a set of cards, each tagged with an RFID sticker. The cards allowed simple answers via smileys (yes/no/not sure, good/bad/neutral etc.), numbers for rating, numbers for minutes of physical activity to log, and a few extra cards with which the participants could actively request services (weather report, system information, activity log, and messages) or interrupt a dialog. The companion used synthesised speech for its utterances. The dialogs are driven by a state machine; roughly, they are represented by a network of utterances whose paths are followed by the dialog manager according to the user’s input or other events. In order to collect rich data, the companion was programmed to take the initiative at certain events and times of the day and to start interactions itself.

Participants were all in the age segment of 50+, the oldest being 76 at the time of the experiment. Some of them took part in all three iterations, some came in later. In that way, we had both “old” and “new” users in iteration 2 and 3:

	Iteration 1	Iteration 2	Iteration 3
Participant 1	yes	yes	yes
Participant 2	yes	yes	yes
Participant 3	yes	yes	yes
Participant 4	—	yes	yes
Participant 5	—	yes	yes
Participant 6	—	—	yes

The data collection resulted in 14 sets of video data, or a total of over 300 video sequences, with an average length between one and two minutes, hence an overall duration of over five hours. Each participant had the system at home for about ten days, in which period we collected 22 videos per person on average, ranging from two to 51. Additionally, half-open interviews were conducted for each dataset, with a length between 30 minutes and over one hour, thus more than seven hours of audio material. After removal of sensitive parts of these data, they will be made available to outside researchers for further studies and analyses.

Analysing this huge amount of data was a major challenge for the project. Research on the data will be continued after the project. From both videos and interviews, we could come, among others, to the following results:

• Not surprisingly, participants found the interactions with the Companion repetitive over time. The simple dialog system did not offer much variety, neither in the type of dialogs nor in their internal design.
• Most participants welcomed the pieces of variable content that the companion had to offer. These were a weather forecast, and the varying “messages” sent by the research team which were formulated as the companion’s own recommendations in the third iteration. On several occasions in the interviews, participants expressed their wish to have more and other services delivered by the companion, their suggestions ranging from music to a timely bus schedule.
• Participants came to the experiment with different expectations. They had and developed their own very individual ways and routines of interacting with the companion. One participant mainly saw in it a device to record her physical activities, and so became somewhat frustrated with other services and dialogs that the companion initiated by its own. Another participant fell into a routine of interactions very quickly and did not change it much, so that she missed some types of interactions and functions altogether. Other participants were more curious and tried to get as much variety as possible out of the device, following the companion-initiated dialogs but also initiating interactions themselves.
• The location chosen for the companion plays a certain role in the style of interaction. While the research team recommended to install it in the entrance hall (in accordance with its function of monitoring the participant’s outdoor activities), some of them had it in a different room. In one case this was a kitchen, in another case a living room. The kitchen was a location where this participant spent considerable time doing everyday household work, the interaction with the companion being interwoven with her other activities. In the hallways, participants passed the companion often on their way between rooms or in and out of the house, but had to stop in these movements to interact. The living room, on the contrary, was a place that the particular participants mostly entered with the intention to interact with the companion.
• Participants experienced the companion as polite and friendly. The reasons for that lay in careful dialog design based on a consistent companion personality. For example, the companion attributed interaction errors to itself, and employed mitigating expressions when it made requests of the human.
• A special case, however, was the question to the user whether they were intending to weigh themselves that day. Despite various conversational strategies employed to make this question acceptable to users, they did not like this topic to be brought up by the companion. Given that companions, in their health-assisting roles, will sometimes have to remind their owners of activities that are not always enjoyable, our findings point to further research that will be needed about the connections between human-companion relationship and compliance with its advice and mentoring.
• Although perceptive and expressive capabilities of our robot were minimal, participants did not seem to miss them. The reason may simply be that they did not expect any of these capabilities from what they characterised, and partly treated, as a machine.

The big challenges that the project SERA has identified for companion research and development are thus situatedness, variability, usefulness, and flexibility. See Reference Architecture for an approach towards key features of a long-term companion. SERA has contributed to making progress in the field of assistive companions and robots. We have shown that:

• It is possible to study companion-human relationships in real life settings. It is, henceforth, possible to include real users and real application environments into research. Realistic experiments are necessary to determine the essential research challenges for companions and to orient further research.
• Users are willing to enter into interaction and even conversation with a robot even though its perceptive and expressive capabilities are limited. Even though they are aware of such limitations, they are ready to “play along”.
• All-time presence of a robot in the home raises challenges to research that risk to be neglected if user studies are done in short lab sessions only: attention and floor management are among these, where the behaviour of a robot also has to be adapted to the specific situation and location in the user’s home - having a robot in the kitchen is different from having it in the hall or in the living room. This will put particular requirements on mobile robots that have the potential to follow users to different locations.
• Participants did not seem to experience the lack of affect recognition and expression as a major shortcoming. They did not expect responsivity to their mood or emotional state. We conclude that, with this system, we remained well on this side of the “uncanny valley” where appearance of an artifact supposedly raises expectations of human-like behaviour that it cannot meet.
• Users tend to be more pragmatic in their relationship toward companions than has been assumed. Their attitude can be resumed as “If it helps me and if it does what I want, I like it!”

For more details on the data analysis results, see also:

Data Analysis Results (SERA Deliverable 1.3b)

Eimler Krämer 2011: How to Design an Engaging Robot - Empirical Results on Determinants of Acceptance and Emotion Attribution in Confrontation with a Robot Rabbit This is a preprint of an article submitted for consideration in the Journal Applied Artificial Intelligence 2011 ©Taylor&Francis.

The data collected in the project can, without exaggeration, be classified as unprecedented. The few long-term studies with virtual and robotic companions done so far had to rely on subjective data (interviews, questionnaires). The existence of a set of video data with older people interacting with a companion in their own homes, over days and even, taking together the phases, weeks, is a resource that will help research to orient towards the users’ needs and the specific requirements for home robotics.

“Social Engagement with Robots and Agents” really happens, we have seen. Despite the limited capabilities of the experimental companion, some of the participants in the field study talked with it. A talking device with some “face”, we conclude, is a strong incentive to take what we call a “conversational stance” toward it. This stance is both a research challenge and a social obligation: it has to be mirrored by socially adequate conversational behaviour on the companion’s side. However, where participants choose not to enter into a conversation, this must also be accepted and catered for in design. Humans have the right to treat the companion as a tool if and when they want.

A further challenge for research is the relevance that participants attributed to the (few) additional functions and services provided (weather report, messages). We conclude that perceived usefulness is key to user satisfaction with companions. Services that are considered helpful and/or entertaining play an important role in increasing the user’s esteem of and building a permanent relationship with the companion.