diaLogic: A Multi-Modal Framework for Automated Team Behavior Modeling Based on Speech Acquisition

This paper presents diaLogic, a humans-in-the-loop system for modeling the behavior of teams during collective problem solving. Team behavior is modeled using multi-modal data about cognition, social interactions, and emotions acquired from speech inputs. The system includes methods for speaker diarization, speaker interaction characterization, speaker emotion recognition, and speech-to-text conversion. Hypotheses about the invariant and differentiated aspects of teams are extracted using the similarities and dissimilarities of their behavior over time. Hypothesis extraction, a novel contribution of this work, uses a method to identify the clauses and concepts in each spoken sentence. Experiments present system performance for a broad set of cases of team behavior during problem solving. The average errors of the various methods are between 6% and 21%. The system can be used in a broad range of applications, from education to team research and therapy.