Demo: Real-time Emotion Recognition through Articulatory Motion and Speech

We demonstrate a real-time emotion recognition system that combines articulatory motion with audio features. Our ear-worn system tracks jaw movements and facial muscle vibrations during speech to achieve robust emotion classification. The system processes three distinct signals: jaw motion, facial muscle vibrations, and bone-borne vibrations, combining them with audio features for emotion detection. We demonstrate the system’s ability to recognize six emotions (happy, sad, anger, fear, disgust, neutral) in real-time with 93% accuracy while robust to body motion artifacts. Our live demonstration allows audience members to observe how different emotional expressions manifest in jaw motion patterns through real-time visualization of emotion classification results.