Reference Architecture

The project SERA proceeded from data collection and analysis to defining a Reference Architecture that captures the essence of the architecture of a collection of systems. The purpose of a Reference Architecture is to provide guidance for the development of architectures, possibly resulting in standards to ensure interoperability. The companion system is intended to be perceived by humans as having certain social skills, in particular those based on awareness of attentiveness, presence of the other. In particular, we addressed the issue of flexibility in interaction, realising that:

• The companion is in need of awareness of the state of attention of the human.
• The human needs to be able to interrupt the companion in the middle of an ongoing dialog, even in the middle of a sentence.
• The companion needs more flexibility in handling communicative behaviour that was already planned and in execution.
• The human is in need of being able to correct the companion when it has made false assumptions, not only regarding the human’s willingness to interact about a certain topic but also regarding the content of what is being said or believed by the companion.

To provide for these skills, the system has the abilities to simultaneously perform a number of interacting perception and action processes and to adjust its behaviour either in a reactive way (on the level of behaviour realisation) or in a deliberate way, on the level of dialog planning. This allows for a mixed-initiative conversation (topic switch initiated by the human) and more natural turn-taking behaviour, a basic social competence. The implementation of the “core architecture” for SERA is not an implementation that is specifically designed for social agents or robots. It rather is a generic set of tools that can be used to help implement about any kind of dialogue system. It is not the architecture that defines the social qualia of an agent - these rather stem from the skills and affordances as perceived by the human that interacts with it.

The software consists of a set of modules that allows one to specify the core part of a dialogue system. This is what we call the “FLIPPER” module. It is essentially an interpreter that processes information state update rules. Another module, “HSMTool”, can help one to design dialogues such as those that were used in the SERA project through visual means, i.e. by drawing graphs. The same module can also be used as a monitoring system that shows the internal state of the dialogue system as the dialogue is executed. The demo system that we made to illustrate the implementation connects these modules with the companion software that simulates conversations with the virtual companion. The realisation of the behaviours of the virtual companion is mediated by modules adapted from the Elckerlyc software.

See Floor Management in SERA’s reference architecture for more details about the demonstration system about some of the essential skills that any interactive system needs to handle, namely turn-taking and topic management. In the interactions recorded during the data collection, we found that the floor management that could be traced in the dialogue was rather crude. In particular, during the first and second version of the companion system, the users could not interrupt or switch to another topic or dialogue.

For more details on the parts of the reference architecture mentioned above and the design process, see:

Implementation of the SERA Core Architecture (SERA Deliverable 3.3)

Heylen et al. 2011: On the Nature of Engineering Social Artificial Companions This is a preprint of an article submitted for consideration in the Journal Applied Artificial Intelligence 2011 ©Taylor&Francis.