With the growing importance of multiagent teamwork, tools that can help humans analyze, evaluate, and understand team behaviors are also becoming increasingly important. To this end, we are creating Isaac, a team analyst agent for post hoc, offline agent-team analysis. ISAAC'S novelty stems from a key design constraint that arises in team analysis: Multiple types of models of team behavior are necessary to analyze different granularities of team events, including agent actions, interactions, and global performance. These heterogeneous team models are automatically acquired by machine learning over teams' external behavior traces, where the specific learning techniques are tailored to the particular model learned. Additionally, ISAAC uses multiple presentation techniques that can aid human understanding of the analyses. This article presents tsa,ac's general conceptual framework and its application in the RoboCup soccer domain, where ISAAC was awarded the RoboCup Scientific Challenge Award.
Multiagent teamwork is an important area of agent research, with a growing number of applications, including multirobotic space missions, virtual environments for training and education, and software agents on the Internet. With the growing importance of teamwork, there is now a critical need for tools to help humans analyze, evaluate, and understand team behaviors. Indeed, in multiagent domains with tens or even hundreds of agents in teams, agent interactions are often highly complex and dynamic, making it difficult for human developers to analyze agent-team behaviors. The problem is further exacerbated in environments where agents are developed by different developers, where even the intended interactions are unpredictable.
Unfortunately, the problem of analyzing team behavior to aid human developers in understanding and improving team performance has largely been unaddressed. Previous work in agent teamwork has largely focused on guiding agents in teamwork (Tambe 1997) but not on its analysis for humans. Agent explanation systems, such as DEBRIeF (Johnson 1994), allow individual agents to explain their actions based on internal state but do not have the means for a team analysis. Recent work on multiagent visualization systems, such as Ndumu et al. (1999), has been motivated by multiagent understandability concerns (similar to ours), but it still leaves analysis of agent actions and interactions to humans.
This article focuses on agents that assist humans to analyze, understand, and improve multiagent team behaviors by (1) locating key aspects of team behaviors that are critical in team success or failure; (2) diagnosing such team behaviors, particularly problematic behaviors; (3) suggesting alternative courses of action; and (4) presenting the relevant information to the user comprehensibly. To accomplish these goals, we have developed an agent called ISAAC. A fundamental design constraint here is that unlike systems that focus on explaining individual agent behaviors (Johnson 1994), team analysts such as ISAAC cannot focus on any single agent or any single perspective or any single granularity (in terms of time scales). Instead, when analyzing teams, multiple perspectives at multiple levels of granularity are important. Thus, although it is sometimes beneficial to analyze the critical actions of single individuals, at other times, it is the collaborative agent interaction that is key in team success or failure and requires analysis, yet at other times, an analysis of the global behavior trends of the entire team is important.
To enable analysis from such multiple perspectives, ISAAC relies on multiple models of team behavior, each covering a different level of granularity of team behavior. More specifically, ISAAC relies on three heterogeneous models that analyze events at three separate levels of granularity: (1) an individual agent action, (2) agent interactions, and (3) overall team behavior. These models are automatically acquired using different methods (inductive learning and pattern matching); indeed, with multiple models, the method of acquisition can be tailored to the model being acquired. …