November, 2019
ToMCAT.
A collaboration between the Information School (INFO), Computer
Science (CS), and Family Studies and Human Development (FSHD)
has been awarded a large grant to develop a theory of mind-based
cognitive architecture for teams (ToMCAT). The grant ($7.5M, for
48 months) is part of the DARPA Artificial Social Intelligence
for Successful Teams
(ASIST)
program.
The PI/Co-PIs collaborating on this project are:
Adarsh Pyarelal (PI),
Kobus Barnard,
Emily Butler,
Clayton Morrison,
Rebecca Sharp,
Mihai Surdeanu, and
Marco Antonio Valenzuela-Escarcega.
Data collection for the project will take place in the
Lang Laboratory,
housed in the
Frances McClelland Institute for Children, Youth and Families
in the Norton School of Family &
Consumer Science.
The goal
of the project is to build artificially intelligent agents that
understand both the social and goal-oriented aspects of teams in
mission-like scenarios (e.g., search-and-rescue missions), and
are able to reason about possible interventions. The agent,
ToMCAT, needs to model human players' affect and beliefs about
the situation and about each other's affect and beliefs (theory
of mind). We will ground this work in extensive measurements of
humans interacting in small teams, that will include audio,
video, eye tracking, electrocardiography (EKG),
electroencephalography (EEG), functional near-infrared
spectroscopy (fNIRS), and self report. The participants will
execute missions within a Minecraft environment with one, two,
three, or four human players interacting with the ToMCAT agent.
Research areas.
One unique aspect of this project is that we will use
simultaneous EEG and fNIRS brain recording from all human team
members to further our understanding of social coordination in
teams. We expect the series of experiments will provide a large
amount of very unique data. ToMCAT's evolving theories of mind
will be implemented using dynamic Bayesian networks interacting
with latent low-level data representation provided by neural
networks. In addition, we will need to understand dialogue as
indicative of affect, plans, and mission goals. Finally, ToMCAT
will need to both understand team plans and also create its own
plans.
Further information
is available
on the project web site
ml4ai.github.io/tomcat
.
This project started Nov 1, 2019. As we move forward, we will
update this website regularly.
